-- Logs begin at Mon 2022-07-18 07:51:17 UTC, end at Mon 2022-07-18 09:05:43 UTC. -- Jul 18 08:23:33.412169 dv-cinder-850142-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 18 08:23:33.420984 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 18 08:23:33.423626 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 18 08:23:33 2022] *** Jul 18 08:23:33.423906 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 18 08:23:33.424083 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 18 08:23:33.424251 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: nodename: dv-cinder-850142-3-iscsi Jul 18 08:23:33.424408 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: machine: x86_64 Jul 18 08:23:33.424560 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: clock source: unix Jul 18 08:23:33.424741 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: pcre jit disabled Jul 18 08:23:33.424897 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: detected number of CPU cores: 4 Jul 18 08:23:33.425053 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: current working directory: / Jul 18 08:23:33.425208 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: detected binary path: /usr/bin/uwsgi-core Jul 18 08:23:33.425372 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: your processes number limit is 47680 Jul 18 08:23:33.425528 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: your memory page size is 4096 bytes Jul 18 08:23:33.425677 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: detected max file descriptor number: 2048 Jul 18 08:23:33.425851 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: lock engine: pthread robust mutexes Jul 18 08:23:33.426177 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: thunder lock: enabled Jul 18 08:23:33.426522 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 18 08:23:33.426757 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 18 08:23:33.466824 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: Python main interpreter initialized at 0x5591c41df9c0 Jul 18 08:23:33.466824 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: python threads support enabled Jul 18 08:23:33.466824 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: your server socket listen backlog is limited to 100 connections Jul 18 08:23:33.466824 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: your mercy for graceful operations on workers is 90 seconds Jul 18 08:23:33.467210 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: mapped 403077 bytes (393 KB) for 2 cores Jul 18 08:23:33.467295 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: *** Operational MODE: preforking *** Jul 18 08:23:33.467516 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: *** uWSGI is running in multiple interpreter mode *** Jul 18 08:23:33.467516 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: spawned uWSGI master process (pid: 80326) Jul 18 08:23:33.467742 dv-cinder-850142-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 18 08:23:33.468426 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: spawned uWSGI worker 1 (pid: 80338, cores: 1) Jul 18 08:23:33.468644 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: spawned uWSGI worker 2 (pid: 80339, cores: 1) Jul 18 08:23:33.468802 dv-cinder-850142-3-iscsi devstack@keystone.service[80326]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 18 08:23:39.007691 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 08:23:39.007691 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: from cryptography.utils import int_from_bytes Jul 18 08:23:39.008502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 08:23:39.008502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: from cryptography.utils import int_from_bytes Jul 18 08:23:39.015335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 08:23:39.015335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: from cryptography.utils import int_from_bytes Jul 18 08:23:39.015999 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 08:23:39.015999 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: from cryptography.utils import int_from_bytes Jul 18 08:23:39.057816 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 08:23:39.057999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 08:23:39.058184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] command line args: [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 08:23:39.058454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 08:23:39.058633 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ================================================================================ {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 08:23:39.058890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] admin_token = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.059140 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] config_dir = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.059328 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.060052 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] config_source = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.060296 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] debug = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.060509 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.061005 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.061097 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.061303 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.061554 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.061787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.061969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_config_append = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.062148 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.062341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_dir = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.062502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_file = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.062740 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.062989 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.063239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.063486 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.063687 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.063785 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 08:23:39.063866 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.063940 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 08:23:39.064101 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.064193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] command line args: [] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 08:23:39.064346 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.064431 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 08:23:39.064547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ================================================================================ {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 08:23:39.064630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.064802 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] admin_token = **** {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.064879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.065036 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.065125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] config_dir = [] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.065203 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.065318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.065403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] config_source = [] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.065505 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.065630 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] debug = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.065740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.065845 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.065976 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.066060 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.066183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.066403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.066484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.066653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.066738 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] publish_errors = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.066882 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.066967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] list_limit = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.067044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] log_config_append = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.067124 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.067224 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.067309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.067430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] log_dir = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.067511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.067586 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] log_file = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.067759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.067848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.067999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] standard_threads = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.068085 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.068232 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.068315 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.068476 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.068554 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.068654 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.068732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.068829 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.068921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] use_journal = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.068999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] use_json = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.069073 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.069165 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] use_stderr = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.069300 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.069383 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] use_syslog = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.069535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.069613 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.069768 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.069865 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.070096 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.070204 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.070331 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.070407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.070477 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.070553 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.070693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.070771 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.070964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.071041 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.071211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.071451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.071609 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.071725 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] publish_errors = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.071805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.071943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.072040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.072198 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.072275 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.072388 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.072471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.072659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.072745 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.072897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.072983 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.073146 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.073229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] standard_threads = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.073388 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.073473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.073626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.073702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.073834 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.073910 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.074035 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] use_journal = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.074121 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.074223 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] use_json = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.074361 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.074439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] use_stderr = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.074594 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] use_syslog = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.074676 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.074847 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.074930 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 08:23:39.075135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.075810 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.075928 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.076180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.076363 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.076448 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.076636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.076719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.076871 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.076948 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.077111 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.077196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.077361 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.077444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.077621 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.077704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.077851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.077932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.078073 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.078180 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.078321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.078455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.078572 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.078720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.078802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.078983 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.079068 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.079246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.079331 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.079495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.079593 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.079755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.079845 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.079990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.080069 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.080229 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.080307 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.080463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.080546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.080694 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.080802 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.080933 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.081075 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.081182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.081326 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.081447 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.081604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.081707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.081859 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.081993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.082114 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.082223 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.082326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.082432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.082558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.082643 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.082790 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.082885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.083031 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.083152 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.083271 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.083349 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.083511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.083599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.083774 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.083868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.084016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.084101 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.084249 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.084332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.084498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.084577 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.084737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.084824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.084982 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.085104 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.085233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.085355 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.085483 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.085609 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.085735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.085860 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.085949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.086113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.086245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.086372 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.086512 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.086638 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.086769 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.086900 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.087031 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.087112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.087272 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.087437 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.087576 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.087729 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.087860 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.087986 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.088120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.088245 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.088377 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.088509 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.088634 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.088759 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.088883 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.089012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.089147 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.089272 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.089272 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.089514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.089643 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.089774 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.089915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.090040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.090161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.090291 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.090416 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.090541 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.090757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.091390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.091515 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.091635 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.091774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.091907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.091907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.092178 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.092309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.092430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.092559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.092677 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.092809 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.092929 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.093058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.093183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.093307 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.093437 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.093558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.093678 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.093811 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.093948 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.094077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.094155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.094215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.094386 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.094520 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.094646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.094646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.094880 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.095015 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.095140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.095271 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.095396 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.095530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.095654 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.095827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.095960 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.096093 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.096225 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.096346 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.096426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.096544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.096671 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.096797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.external = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.096922 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.097046 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.097170 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.097304 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.097437 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.097558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.097684 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.097809 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.password = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.097939 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.098069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] auth.token = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.098199 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.098324 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.098455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.098580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.098704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.098834 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.098976 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.099102 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.099226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.099350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.099475 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.099599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.099748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.099868 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.099993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.100117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.100241 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.100371 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.100527 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.100660 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.100806 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.100933 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.101059 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.101183 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.101308 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.101308 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.101560 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.101686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.101811 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.101936 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.102067 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.102193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.102326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.102326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.102582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.102582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.102825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.102956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] auth.external = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.103090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.103216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.103340 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.103466 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.103593 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.103743 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.103879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.104005 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] auth.password = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.104131 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.104266 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] auth.token = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.104391 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.104521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.104655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.104781 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.104911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.105100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.105222 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.105348 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.105549 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.105676 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.105802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.105929 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.106060 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.106191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.106321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.106997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] credential.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.107133 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.107261 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.107387 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.107513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.107646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.107786 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.107908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.108033 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.108158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.108292 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.108436 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.108436 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.108630 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.108764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.108764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.109025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.109154 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.109286 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.109407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.109528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.109655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.109783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.109904 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.110024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.110155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.110279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.110399 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.110399 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.110635 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.110755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.110875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.110993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.111114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.111242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.111363 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] federation.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.111490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.111612 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.111794 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.111916 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.112051 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.112183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.112183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.112438 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.112563 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.112689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.112822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.112948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.113080 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.113205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.113329 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.113454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.113595 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.113723 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.113847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.113972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.114093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.114213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.114346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.114472 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.114597 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.114722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.114847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.114972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.115103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.115238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.115363 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.115487 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.115612 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.115763 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.115896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.116025 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.116146 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.116266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.116386 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.116505 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.116633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.116756 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.116882 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.117020 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.117143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.117262 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.117390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.117511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.117631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.117754 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.117874 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.117993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.118121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.118242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.118362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.118482 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.118862 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.119054 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.119191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.119320 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.119441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.119561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.119731 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.119853 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.119990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.120115 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.120239 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.120364 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.120488 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.120612 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.120753 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.120880 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.121005 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.121125 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.121244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.121364 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.121495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.121620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.121746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.121876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.122023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.122215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.122346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.122476 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.122605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.122732 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.122858 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.122992 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.123119 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.123245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.123370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.123495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.123619 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.123765 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.123888 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.124009 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.124130 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.124261 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.124382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.124510 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.124637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.124758 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.124758 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.124758 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.125089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.125218 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.125339 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.125459 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.125579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.125699 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.125699 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.125941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.125941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.126171 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.126291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.126411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.126531 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.126659 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.126780 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.126900 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.127020 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.127148 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.127270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.127398 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.127524 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.127654 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.127806 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.127932 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.128056 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.128192 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.128316 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.128441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.128565 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.128689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.128815 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.128946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.129070 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.129195 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.129335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.129461 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.129586 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.129723 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.129848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.129973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.130098 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.130220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.130340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.130540 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.130723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.130846 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.130977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.131103 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.131239 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.131329 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.131459 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.131579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.131714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.131845 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.131967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.132096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.132217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.132344 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.132474 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.132594 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.132714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.132842 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] role.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.132973 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.133093 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] role.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.133213 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.133334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.133454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.133582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.133704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.133824 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.133945 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.134065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.134185 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.134314 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.134436 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.134567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.134688 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.134809 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.134928 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.135055 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.135177 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.135298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.135418 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.135539 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.135685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.135808 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.135928 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.136049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.136178 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.136313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.136443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.136565 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.136686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.136686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.136912 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.137033 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] resource.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.137161 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.137283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.137404 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.137525 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.137691 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.137875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.138061 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.138247 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.138459 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.138584 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.138706 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.138827 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.138957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.139080 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.139201 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.139322 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.139443 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.139565 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.139744 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.139868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] role.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.140008 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.140134 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] role.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.140254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.140379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.140512 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.140639 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.140765 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.140890 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.141016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.141141 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.141273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.141399 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.141523 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.141658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.141784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.141784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.141784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.142138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.142263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.142386 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.142513 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.142633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.142771 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.142898 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.143022 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.143146 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.143270 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.143405 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.143538 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.143680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.143804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.143931 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.144056 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.144056 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.144303 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.144303 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.144538 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.144538 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.144774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.144907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.145043 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.145168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.145294 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.145418 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.145543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.145677 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.145800 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.145800 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.146027 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.146148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.146269 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.connection = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.146402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.146533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.146673 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.146795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.146916 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.147036 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.147158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.147278 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.147410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.147533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.147689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.147813 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.147935 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.148056 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.148188 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.148310 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.148441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.148563 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.148686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.148807 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.148940 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] token.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.149062 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.149184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.149311 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.149433 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.149562 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.149684 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.149805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.149926 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.150047 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.150182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.150313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.150435 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.150556 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.150679 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.150799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.150799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80339) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 08:23:39.151036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.151036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.151036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.151036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.151036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.151036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.151036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.151036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.151036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.152095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.152095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.connection = **** {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.152095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.152095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.152095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.152095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.152095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.152095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.152095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.153125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.153125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.153125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.153125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.153125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.153125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.153125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.153125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.153125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.154765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.154765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 08:23:39.154765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 08:23:39.178382 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.178600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.178830 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.180742 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.180973 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.181199 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.187411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.190020 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.215332 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.215556 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.217230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.217461 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.363437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.367521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.415344 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.415629 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.417579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.417817 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.483446 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.483725 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.483862 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.484000 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.484146 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.484294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.484426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.484550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.484674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.484799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.484936 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.484936 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.510224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.510579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.510710 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.510854 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.510995 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.511119 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.511242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.511364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.511496 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.511622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.511762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.511886 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.512010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.512143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.523822 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.524343 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.538517 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.538751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.538951 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.539173 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.539388 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.539599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.539758 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.539889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.540024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.540163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.540357 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.540612 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.540796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.541016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80339) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 08:23:39.558256 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.558498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 08:23:39.558640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.558785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.558925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 08:23:39.559082 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 08:23:39.559242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.559377 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.559556 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 08:23:39.560205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.560479 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.560740 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.560897 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.561170 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.561428 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.561682 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.571958 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.572241 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.572512 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.572796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.572915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.573125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.573389 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.573664 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.581216 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.582615 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.584537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.584823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.585102 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.585509 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.585791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.585914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.586060 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.586193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.586321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.586472 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.586593 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.586868 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.586949 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.587141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.587221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.587394 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.587483 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.587659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.587951 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.588215 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.588483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.588748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.614681 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.616291 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.616574 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.616961 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.618832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.619113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.622656 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.622935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.623392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.625515 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.625798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.626071 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.630705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.630991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.633420 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.633697 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.635628 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.635938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.636220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.636501 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.636780 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.637057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.637327 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.637677 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.638922 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.639205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.639484 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.639785 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.640057 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.640325 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.640588 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.640851 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.656614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.656897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.659954 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.660228 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.661434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.664740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.665357 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.665655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.668338 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.668627 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.671263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.674650 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.674902 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.678280 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.678672 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.678972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.679256 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.679538 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.679855 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.680143 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.682038 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.682334 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.682614 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.682892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.683172 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.683454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.699869 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.700233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.700503 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.700776 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.704038 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.704426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.704692 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.704961 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.708724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.710431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.710704 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.712870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.713802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.714092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.714379 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.714575 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.714658 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.714852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.714934 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.718482 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.718758 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.719037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.719326 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.719598 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.728524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.728804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.729084 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.729365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.729647 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.729928 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.730199 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.734086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.734362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.734644 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.734920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.735204 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.735479 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.735769 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.751262 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.754503 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.756419 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.757833 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.759438 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.759675 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.761113 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.761389 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.761667 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.764042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.765669 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.767298 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.767572 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.767879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.769630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.771282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.774583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.774872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.775156 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.775244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.775430 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.776851 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.781313 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.781602 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.781878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.782151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.785342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.785635 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.785916 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.791337 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.791626 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.791922 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.793756 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80339) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.794073 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.794354 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.794630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.795087 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.795379 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.795659 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.795950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.796224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.796489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.796769 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.797041 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.797343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.797585 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80339) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.800508 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 08:23:39.800795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.801079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.801351 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.801619 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.801891 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.802157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.802433 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.802701 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.802965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.803238 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.803501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.803790 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.804050 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 08:23:39.837281 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5591c41df9c0 pid: 80339 (default app) Jul 18 08:23:39.840233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97774e-52dc-48b5-bee8-20dcf00770aa None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:39.840534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97774e-52dc-48b5-bee8-20dcf00770aa None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:39.840855 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97774e-52dc-48b5-bee8-20dcf00770aa None None] PATH_INFO: `/v3` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:39.844532 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5591c41df9c0 pid: 80338 (default app) Jul 18 08:23:39.845266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Mon Jul 18 08:23:39 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:23:48.548799 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad883bd0-c2bb-4ba7-9109-2e8d50f336dc None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:48.549344 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad883bd0-c2bb-4ba7-9109-2e8d50f336dc None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:48.549344 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad883bd0-c2bb-4ba7-9109-2e8d50f336dc None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:48.553888 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:23:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:23:48.560558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ba41c-a749-43a4-ae3d-3515d9b779ac None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:48.561083 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ba41c-a749-43a4-ae3d-3515d9b779ac None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:48.561083 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ba41c-a749-43a4-ae3d-3515d9b779ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:48.629609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: INFO dbcounter [None req-650ba41c-a749-43a4-ae3d-3515d9b779ac None None] Registered counter for database keystone Jul 18 08:23:48.651138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG oslo_db.sqlalchemy.engines [None req-650ba41c-a749-43a4-ae3d-3515d9b779ac None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80339) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 08:23:48.657574 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writer thread running {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 08:23:48.838324 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG passlib.handlers.bcrypt [None req-650ba41c-a749-43a4-ae3d-3515d9b779ac None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80339) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 08:23:48.838749 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG passlib.handlers.bcrypt [None req-650ba41c-a749-43a4-ae3d-3515d9b779ac None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80339) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 08:23:48.894826 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-650ba41c-a749-43a4-ae3d-3515d9b779ac None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:23:48.896566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-650ba41c-a749-43a4-ae3d-3515d9b779ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:23:49.022357 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:23:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 18 08:23:49.178335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad883bd0-c2bb-4ba7-9109-2e8d50f336dc None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:49.247129 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: INFO dbcounter [None req-ad883bd0-c2bb-4ba7-9109-2e8d50f336dc None None] Registered counter for database keystone Jul 18 08:23:49.266710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG oslo_db.sqlalchemy.engines [None req-ad883bd0-c2bb-4ba7-9109-2e8d50f336dc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80338) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 08:23:49.272940 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writer thread running {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 08:23:49.391795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad883bd0-c2bb-4ba7-9109-2e8d50f336dc None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:49.396097 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f923fb7-467d-45ce-88d8-982acd54ae21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:49.396771 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f923fb7-467d-45ce-88d8-982acd54ae21 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:49.397060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f923fb7-467d-45ce-88d8-982acd54ae21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:49.397439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f923fb7-467d-45ce-88d8-982acd54ae21 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:49.406594 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f923fb7-467d-45ce-88d8-982acd54ae21 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:49.457579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3f923fb7-467d-45ce-88d8-982acd54ae21 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 08:23:49.457922 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f923fb7-467d-45ce-88d8-982acd54ae21 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:49.468677 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Mon Jul 18 08:23:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 08:23:49.474837 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ba41c-a749-43a4-ae3d-3515d9b779ac None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:49.489541 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ba41c-a749-43a4-ae3d-3515d9b779ac None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:49.494401 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298beef-caaa-47a6-9bdd-0f6a02d073f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:49.495521 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298beef-caaa-47a6-9bdd-0f6a02d073f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:49.495844 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298beef-caaa-47a6-9bdd-0f6a02d073f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:49.496124 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298beef-caaa-47a6-9bdd-0f6a02d073f2 admin admin] PATH_INFO: `/v3/projects/eedba945f8d24629a712fbf156a2209f` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:49.500432 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298beef-caaa-47a6-9bdd-0f6a02d073f2 admin admin] RBAC: Authorizing `identity:get_project(project_id=eedba945f8d24629a712fbf156a2209f)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:49.551312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7298beef-caaa-47a6-9bdd-0f6a02d073f2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 08:23:49.551656 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7298beef-caaa-47a6-9bdd-0f6a02d073f2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:49.556525 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Mon Jul 18 08:23:49 2022] GET /identity/v3/projects/eedba945f8d24629a712fbf156a2209f => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:23:51.568619 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac64aa-1eaf-42f1-89ff-24cedb88ba88 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:51.569151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac64aa-1eaf-42f1-89ff-24cedb88ba88 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:51.569151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac64aa-1eaf-42f1-89ff-24cedb88ba88 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:51.573331 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:23:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:23:51.579065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c82f1c-6205-4ac4-b099-9961486836dd None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:51.579470 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c82f1c-6205-4ac4-b099-9961486836dd None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:51.579623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c82f1c-6205-4ac4-b099-9961486836dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:51.598826 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-51c82f1c-6205-4ac4-b099-9961486836dd None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:23:51.600543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-51c82f1c-6205-4ac4-b099-9961486836dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:23:51.609154 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:23:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 08:23:51.632219 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac64aa-1eaf-42f1-89ff-24cedb88ba88 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:51.644873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac64aa-1eaf-42f1-89ff-24cedb88ba88 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:51.649097 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35eea1-d253-43de-8c52-90877318c24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:51.649732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35eea1-d253-43de-8c52-90877318c24e admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:51.650015 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35eea1-d253-43de-8c52-90877318c24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:51.650293 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35eea1-d253-43de-8c52-90877318c24e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:51.658183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f35eea1-d253-43de-8c52-90877318c24e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:51.660277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f35eea1-d253-43de-8c52-90877318c24e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:51.669528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Mon Jul 18 08:23:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 08:23:51.675258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c82f1c-6205-4ac4-b099-9961486836dd None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:51.685577 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c82f1c-6205-4ac4-b099-9961486836dd None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:51.690409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4602e-4bab-4aeb-8821-d23fd0031bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:51.691268 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4602e-4bab-4aeb-8821-d23fd0031bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:51.691585 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4602e-4bab-4aeb-8821-d23fd0031bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:51.691890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4602e-4bab-4aeb-8821-d23fd0031bc8 admin admin] PATH_INFO: `/v3/users/93ad24fe28ec40dbbca364cbdb34972d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:51.693296 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4602e-4bab-4aeb-8821-d23fd0031bc8 admin admin] RBAC: Authorizing `identity:get_user(user_id=93ad24fe28ec40dbbca364cbdb34972d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:51.695160 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4602e-4bab-4aeb-8821-d23fd0031bc8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:51.696845 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Mon Jul 18 08:23:51 2022] GET /identity/v3/users/93ad24fe28ec40dbbca364cbdb34972d => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:23:53.772432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23384b-2af1-4f26-9480-df801040511b None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:53.772950 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23384b-2af1-4f26-9480-df801040511b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:53.773035 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23384b-2af1-4f26-9480-df801040511b None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:53.774340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:23:53.780323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48e5d5-cdeb-497b-8e61-7d3f0dbdbe29 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:53.780779 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48e5d5-cdeb-497b-8e61-7d3f0dbdbe29 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:53.781063 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48e5d5-cdeb-497b-8e61-7d3f0dbdbe29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:53.797748 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-dc48e5d5-cdeb-497b-8e61-7d3f0dbdbe29 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:23:53.798955 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-dc48e5d5-cdeb-497b-8e61-7d3f0dbdbe29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:23:53.803967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4fb21-8f6a-417d-a0a4-48bc3bd0f646 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:53.804255 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4fb21-8f6a-417d-a0a4-48bc3bd0f646 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:53.804534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4fb21-8f6a-417d-a0a4-48bc3bd0f646 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:53.805876 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 6/11] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:23:53.810917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb19493-08bb-4464-8154-d0230d61a538 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:53.811205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb19493-08bb-4464-8154-d0230d61a538 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:53.811484 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb19493-08bb-4464-8154-d0230d61a538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:53.853391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG passlib.handlers.bcrypt [None req-8fb19493-08bb-4464-8154-d0230d61a538 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80338) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 08:23:53.853805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG passlib.handlers.bcrypt [None req-8fb19493-08bb-4464-8154-d0230d61a538 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80338) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 08:23:53.883017 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 6/12] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:23:53.899532 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-8fb19493-08bb-4464-8154-d0230d61a538 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:23:53.901161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8fb19493-08bb-4464-8154-d0230d61a538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:23:53.907158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48e5d5-cdeb-497b-8e61-7d3f0dbdbe29 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:53.931752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:23:54.004447 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:23:54.015321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48e5d5-cdeb-497b-8e61-7d3f0dbdbe29 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:54.026698 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb19493-08bb-4464-8154-d0230d61a538 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:54.050005 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ca505-d124-4f33-8e2e-b6d36b446ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:54.050825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ca505-d124-4f33-8e2e-b6d36b446ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:54.051116 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ca505-d124-4f33-8e2e-b6d36b446ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:54.051407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ca505-d124-4f33-8e2e-b6d36b446ba1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:54.064548 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ca505-d124-4f33-8e2e-b6d36b446ba1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:23:54.076622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-252ca505-d124-4f33-8e2e-b6d36b446ba1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:23:54.076622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:23:54.076622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:23:54.076622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:23:54.076622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:23:54.076622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:23:54.076622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:23:54.076622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:23:54.076622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:23:54.076622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:23:54.076622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:23:54.076622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:23:54.076622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:23:54.076622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:23:54.076622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:23:54.076622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:23:54.077490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:23:54.077490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:23:54.077490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:23:54.077490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:23:54.077490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:23:54.077490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:23:54.077490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:23:54.077490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:23:54.077490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:23:54.077490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:23:54.077490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:23:54.077490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:23:54.077490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:23:54.077490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:23:54.077490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:23:54.077490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:23:54.078334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:23:54.078334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:23:54.078334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:23:54.078334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:23:54.078334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:23:54.078334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:23:54.078334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:23:54.078334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:23:54.078334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:23:54.078334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:23:54.078334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:23:54.078334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:23:54.078334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 7/14] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:23:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:23:54.083827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ca505-d124-4f33-8e2e-b6d36b446ba1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:54.129808 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb19493-08bb-4464-8154-d0230d61a538 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:54.132137 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:23:54.169195 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:23:54.169388 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1975f09-cd12-4fcc-b5a9-205890290fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:54.170117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1975f09-cd12-4fcc-b5a9-205890290fed None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:54.170403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1975f09-cd12-4fcc-b5a9-205890290fed None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:54.170885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1975f09-cd12-4fcc-b5a9-205890290fed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:54.187801 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ca505-d124-4f33-8e2e-b6d36b446ba1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:54.223653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449a2d7-e6ae-455b-aece-15aaaa43bdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:54.224344 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449a2d7-e6ae-455b-aece-15aaaa43bdcf None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:54.224624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449a2d7-e6ae-455b-aece-15aaaa43bdcf None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:54.224897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449a2d7-e6ae-455b-aece-15aaaa43bdcf None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:54.225651 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449a2d7-e6ae-455b-aece-15aaaa43bdcf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:23:54.225956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449a2d7-e6ae-455b-aece-15aaaa43bdcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:54.227349 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449a2d7-e6ae-455b-aece-15aaaa43bdcf None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:54.239959 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1975f09-cd12-4fcc-b5a9-205890290fed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:54.240108 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 8/15] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:23:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:23:54.241248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1975f09-cd12-4fcc-b5a9-205890290fed None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:54.245726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449a2d7-e6ae-455b-aece-15aaaa43bdcf None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:54.339276 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 8/16] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:23:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 08:23:54.345417 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1975f09-cd12-4fcc-b5a9-205890290fed None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:54.345914 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449a2d7-e6ae-455b-aece-15aaaa43bdcf None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:54.349039 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:23:54.381131 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7df44-b5fe-4a16-a5d2-5f1880fa1ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:54.382012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7df44-b5fe-4a16-a5d2-5f1880fa1ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:54.382305 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7df44-b5fe-4a16-a5d2-5f1880fa1ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:54.382586 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7df44-b5fe-4a16-a5d2-5f1880fa1ea0 None admin] PATH_INFO: `/v3/users/93ad24fe28ec40dbbca364cbdb34972d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:54.383845 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7df44-b5fe-4a16-a5d2-5f1880fa1ea0 None admin] RBAC: Authorizing `identity:get_user(user_id=93ad24fe28ec40dbbca364cbdb34972d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:54.385309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7df44-b5fe-4a16-a5d2-5f1880fa1ea0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:54.387477 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 9/17] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:23:54 2022] GET /identity/v3/users/93ad24fe28ec40dbbca364cbdb34972d => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:23:54.392865 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7df44-b5fe-4a16-a5d2-5f1880fa1ea0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:54.408028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:23:54.450739 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1975f09-cd12-4fcc-b5a9-205890290fed None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:54.484952 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13bd9c-6eff-40a9-9fff-de808b0bbbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:54.485750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13bd9c-6eff-40a9-9fff-de808b0bbbbe None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:54.486037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13bd9c-6eff-40a9-9fff-de808b0bbbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:54.486314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13bd9c-6eff-40a9-9fff-de808b0bbbbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:54.498542 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7df44-b5fe-4a16-a5d2-5f1880fa1ea0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:54.507101 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13bd9c-6eff-40a9-9fff-de808b0bbbbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:54.508690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13bd9c-6eff-40a9-9fff-de808b0bbbbe None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:54.530637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-da13bd9c-6eff-40a9-9fff-de808b0bbbbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:23:54.530637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:23:54.530637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:23:54.530637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:23:54.530637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:23:54.530637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:23:54.530637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:23:54.530637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:23:54.530637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:23:54.530637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:23:54.530637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:23:54.530637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:23:54.530637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:23:54.530637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:23:54.530637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:23:54.531513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:23:54.531513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:23:54.531513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:23:54.531513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:23:54.531513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:23:54.531513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:23:54.531513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:23:54.531513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:23:54.531513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:23:54.531513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:23:54.531513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:23:54.531513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:23:54.531513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:23:54.531513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:23:54.531513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:23:54.531513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:23:54.531513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:23:54.532574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:23:54.532574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:23:54.532574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:23:54.532574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:23:54.532574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 9/18] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:23:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:23:54.534775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1149a-8eeb-421e-9d08-2e247081cd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:54.535578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1149a-8eeb-421e-9d08-2e247081cd8b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:54.535902 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1149a-8eeb-421e-9d08-2e247081cd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:54.536186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1149a-8eeb-421e-9d08-2e247081cd8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:54.537717 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1149a-8eeb-421e-9d08-2e247081cd8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:54.537817 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13bd9c-6eff-40a9-9fff-de808b0bbbbe None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:54.539200 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1149a-8eeb-421e-9d08-2e247081cd8b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:54.540871 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 10/19] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:23:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:23:54.549531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1149a-8eeb-421e-9d08-2e247081cd8b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:54.598648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:23:54.634513 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:23:54.643495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13bd9c-6eff-40a9-9fff-de808b0bbbbe None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:54.653711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1149a-8eeb-421e-9d08-2e247081cd8b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:54.677807 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63480d17-549f-40e4-8d8a-851c52ae1744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:54.678473 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63480d17-549f-40e4-8d8a-851c52ae1744 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:54.678758 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63480d17-549f-40e4-8d8a-851c52ae1744 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:54.679033 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63480d17-549f-40e4-8d8a-851c52ae1744 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:54.679789 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63480d17-549f-40e4-8d8a-851c52ae1744 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:23:54.680103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63480d17-549f-40e4-8d8a-851c52ae1744 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:54.681480 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63480d17-549f-40e4-8d8a-851c52ae1744 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:54.688691 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b33e0f-e9f4-413b-85c7-bd8d60962dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:54.689352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b33e0f-e9f4-413b-85c7-bd8d60962dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:54.689639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b33e0f-e9f4-413b-85c7-bd8d60962dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:54.689920 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b33e0f-e9f4-413b-85c7-bd8d60962dd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:54.690762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b33e0f-e9f4-413b-85c7-bd8d60962dd6 None admin] RBAC: Adding query filter params (role.id=467d49b8a0d3435aaba58464cc8ef85c, scope.domain.id=default, user.id=93ad24fe28ec40dbbca364cbdb34972d) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:23:54.691075 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b33e0f-e9f4-413b-85c7-bd8d60962dd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:54.692553 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b33e0f-e9f4-413b-85c7-bd8d60962dd6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:54.699226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 10/20] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:23:54.699401 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 11/21] 10.222.0.57 () {56 vars in 1322 bytes} [Mon Jul 18 08:23:54 2022] GET /identity/v3/role_assignments?user.id=93ad24fe28ec40dbbca364cbdb34972d&scope.domain.id=default&role.id=467d49b8a0d3435aaba58464cc8ef85c => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:23:57.084189 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b4cee-50b5-42c6-93ba-dc3edb44ec28 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:57.085201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b4cee-50b5-42c6-93ba-dc3edb44ec28 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:57.085708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b4cee-50b5-42c6-93ba-dc3edb44ec28 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:57.087261 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 11/22] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:23:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:23:57.093620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da657157-fe21-47a3-8227-cf2ad6022540 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:57.094502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da657157-fe21-47a3-8227-cf2ad6022540 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:57.094999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da657157-fe21-47a3-8227-cf2ad6022540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:57.120693 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-da657157-fe21-47a3-8227-cf2ad6022540 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:23:57.122848 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-da657157-fe21-47a3-8227-cf2ad6022540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:23:57.214268 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 12/23] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:23:57.251250 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b4cee-50b5-42c6-93ba-dc3edb44ec28 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:57.276366 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82976799-563d-4cf5-9504-a2f1a1487a47 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:57.277001 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82976799-563d-4cf5-9504-a2f1a1487a47 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:57.278136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82976799-563d-4cf5-9504-a2f1a1487a47 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:57.280262 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 13/24] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:23:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:23:57.285839 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bcb62-6ed0-4815-96a8-7e19d927b2c1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:57.286414 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bcb62-6ed0-4815-96a8-7e19d927b2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:57.286890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bcb62-6ed0-4815-96a8-7e19d927b2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:57.307391 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-832bcb62-6ed0-4815-96a8-7e19d927b2c1 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:23:57.310836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-832bcb62-6ed0-4815-96a8-7e19d927b2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:23:57.413195 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 14/25] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:23:57.449029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b4cee-50b5-42c6-93ba-dc3edb44ec28 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:57.449850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bcb62-6ed0-4815-96a8-7e19d927b2c1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:57.456857 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:23:57.480915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:23:57.517129 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b57138-ea12-406d-9410-fbcdf7985a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:57.518367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b57138-ea12-406d-9410-fbcdf7985a3c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:57.518846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b57138-ea12-406d-9410-fbcdf7985a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:57.519321 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b57138-ea12-406d-9410-fbcdf7985a3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:57.538248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b57138-ea12-406d-9410-fbcdf7985a3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:57.540243 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b57138-ea12-406d-9410-fbcdf7985a3c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:57.564693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-e9b57138-ea12-406d-9410-fbcdf7985a3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:23:57.564693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:23:57.564693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:23:57.564693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:23:57.564693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:23:57.564693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:23:57.564693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:23:57.564693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:23:57.564693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:23:57.564693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:23:57.564693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:23:57.564693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:23:57.564693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:23:57.564693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:23:57.564693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:23:57.565573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:23:57.565573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:23:57.565573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:23:57.565573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:23:57.565573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:23:57.565573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:23:57.565573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:23:57.565573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:23:57.565573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:23:57.565573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:23:57.565573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:23:57.565573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:23:57.565573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:23:57.565573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:23:57.565573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:23:57.565573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:23:57.565573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:23:57.566364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:23:57.566364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:23:57.566364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:23:57.566364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:23:57.568364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 12/26] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:23:57.575416 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b57138-ea12-406d-9410-fbcdf7985a3c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:57.691512 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b57138-ea12-406d-9410-fbcdf7985a3c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:57.730375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e859ba-52b4-4bca-a79d-f4d63462c08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:57.732488 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e859ba-52b4-4bca-a79d-f4d63462c08b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:57.732976 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e859ba-52b4-4bca-a79d-f4d63462c08b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:57.733446 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e859ba-52b4-4bca-a79d-f4d63462c08b None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:57.734428 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e859ba-52b4-4bca-a79d-f4d63462c08b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:23:57.734932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e859ba-52b4-4bca-a79d-f4d63462c08b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:57.736568 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e859ba-52b4-4bca-a79d-f4d63462c08b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:57.751301 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 13/27] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:23:57.753103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40764a78-3436-42a0-8130-e5e34fcd2153 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:57.753628 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40764a78-3436-42a0-8130-e5e34fcd2153 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:57.754097 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40764a78-3436-42a0-8130-e5e34fcd2153 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:57.755646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 14/28] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:23:57.757015 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40764a78-3436-42a0-8130-e5e34fcd2153 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:57.758778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bcb62-6ed0-4815-96a8-7e19d927b2c1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:57.790460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:23:57.869728 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88c20c-6bb0-47b3-8c45-251b2dbd53b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:57.870569 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88c20c-6bb0-47b3-8c45-251b2dbd53b7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:57.870858 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88c20c-6bb0-47b3-8c45-251b2dbd53b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:57.871195 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88c20c-6bb0-47b3-8c45-251b2dbd53b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:57.887898 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88c20c-6bb0-47b3-8c45-251b2dbd53b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:23:57.891732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40764a78-3436-42a0-8130-e5e34fcd2153 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:57.897504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-9a88c20c-6bb0-47b3-8c45-251b2dbd53b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:23:57.897504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:23:57.897504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:23:57.897504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:23:57.897504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:23:57.897504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:23:57.897504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:23:57.897504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:23:57.897504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:23:57.897504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:23:57.897504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:23:57.897504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:23:57.897504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:23:57.897504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:23:57.897504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:23:57.897504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:23:57.898471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:23:57.898471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:23:57.898471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:23:57.898471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:23:57.898471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:23:57.898471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:23:57.898471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:23:57.898471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:23:57.898471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:23:57.898471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:23:57.898471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:23:57.898471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:23:57.898471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:23:57.898471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:23:57.898471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:23:57.898471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:23:57.899242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:23:57.899242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:23:57.899242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:23:57.899242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:23:57.899242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:23:57.899242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:23:57.899242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:23:57.899242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:23:57.899242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:23:57.899242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:23:57.899242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:23:57.899242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:23:57.907198 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 15/29] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:23:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 453 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:23:57.909190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc189f-85cf-4ba6-ab44-86348e24e7cc None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:57.909821 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc189f-85cf-4ba6-ab44-86348e24e7cc None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:57.910311 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc189f-85cf-4ba6-ab44-86348e24e7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:57.931145 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c4dc189f-85cf-4ba6-ab44-86348e24e7cc None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:23:57.932591 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c4dc189f-85cf-4ba6-ab44-86348e24e7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:23:57.936289 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3900d-e324-4c98-8c8a-bb855220af54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:57.937079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3900d-e324-4c98-8c8a-bb855220af54 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:57.937467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3900d-e324-4c98-8c8a-bb855220af54 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:57.937822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3900d-e324-4c98-8c8a-bb855220af54 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:57.938985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3900d-e324-4c98-8c8a-bb855220af54 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:57.940853 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3900d-e324-4c98-8c8a-bb855220af54 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:58.008475 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:23:58.020163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 16/30] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:23:58.021686 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47562a-aa6a-4256-b440-ff265b4d94b9 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:58.021975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47562a-aa6a-4256-b440-ff265b4d94b9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:58.022262 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47562a-aa6a-4256-b440-ff265b4d94b9 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:58.027872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 17/31] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:23:58.027872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47562a-aa6a-4256-b440-ff265b4d94b9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:58.188226 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47562a-aa6a-4256-b440-ff265b4d94b9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:58.234314 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cb28a-32e3-4254-a2d3-b289fae606fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:58.235257 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cb28a-32e3-4254-a2d3-b289fae606fe None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:58.235764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cb28a-32e3-4254-a2d3-b289fae606fe None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:58.236197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cb28a-32e3-4254-a2d3-b289fae606fe None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:58.237117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cb28a-32e3-4254-a2d3-b289fae606fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:23:58.237634 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cb28a-32e3-4254-a2d3-b289fae606fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:58.239247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cb28a-32e3-4254-a2d3-b289fae606fe None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:58.251897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 18/32] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:23:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:23:58.253613 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fca454-c0e6-4929-9e8d-3365c571d08a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:58.254165 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fca454-c0e6-4929-9e8d-3365c571d08a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:58.254663 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fca454-c0e6-4929-9e8d-3365c571d08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:58.275506 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d2fca454-c0e6-4929-9e8d-3365c571d08a None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:23:58.277219 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d2fca454-c0e6-4929-9e8d-3365c571d08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:23:58.347749 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 15/33] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 18 08:23:57 2022] POST /identity/v3/projects => generated 301 bytes in 587 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:23:58.347749 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3900d-e324-4c98-8c8a-bb855220af54 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:58.356677 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:23:58.394238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 19/34] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:23:58.395552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fca454-c0e6-4929-9e8d-3365c571d08a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:58.520044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:23:58.561328 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3900d-e324-4c98-8c8a-bb855220af54 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:58.596262 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fca454-c0e6-4929-9e8d-3365c571d08a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:58.620881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27ceff-7997-418b-82a1-833a1b9f034b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:58.622023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27ceff-7997-418b-82a1-833a1b9f034b None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:58.622516 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27ceff-7997-418b-82a1-833a1b9f034b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:58.622791 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27ceff-7997-418b-82a1-833a1b9f034b None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:58.623551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27ceff-7997-418b-82a1-833a1b9f034b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:58.625163 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27ceff-7997-418b-82a1-833a1b9f034b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:58.659430 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b7ba9-51c0-4a14-aa98-d3d50464a2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:58.660684 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b7ba9-51c0-4a14-aa98-d3d50464a2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:58.660973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b7ba9-51c0-4a14-aa98-d3d50464a2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:58.661257 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b7ba9-51c0-4a14-aa98-d3d50464a2a5 None admin] PATH_INFO: `/v3/users/93ad24fe28ec40dbbca364cbdb34972d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:58.662579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b7ba9-51c0-4a14-aa98-d3d50464a2a5 None admin] RBAC: Authorizing `identity:get_user(user_id=93ad24fe28ec40dbbca364cbdb34972d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:58.664139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b7ba9-51c0-4a14-aa98-d3d50464a2a5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:58.670722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 20/35] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:23:58 2022] GET /identity/v3/users/93ad24fe28ec40dbbca364cbdb34972d => generated 247 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:23:58.670722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b7ba9-51c0-4a14-aa98-d3d50464a2a5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:58.720503 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 16/36] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Jul 18 08:23:58 2022] POST /identity/v3/roles => generated 225 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:23:58.722152 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2347a9c-1adf-4371-ad5d-16def8cabe34 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:58.722518 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2347a9c-1adf-4371-ad5d-16def8cabe34 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:58.722880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2347a9c-1adf-4371-ad5d-16def8cabe34 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:58.724979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 17/37] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:23:58.732658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2347a9c-1adf-4371-ad5d-16def8cabe34 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:58.746824 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:23:58.845431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b7ba9-51c0-4a14-aa98-d3d50464a2a5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:58.890623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b0181-c864-4e60-b97f-e7da73976c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:58.891317 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b0181-c864-4e60-b97f-e7da73976c57 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:58.891610 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b0181-c864-4e60-b97f-e7da73976c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:58.891930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b0181-c864-4e60-b97f-e7da73976c57 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:58.892703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b0181-c864-4e60-b97f-e7da73976c57 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:58.894121 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b0181-c864-4e60-b97f-e7da73976c57 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:58.908379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2347a9c-1adf-4371-ad5d-16def8cabe34 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:58.950939 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a127dc-e660-4e2e-b81f-f6c7df63f443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:58.951864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a127dc-e660-4e2e-b81f-f6c7df63f443 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:58.952164 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a127dc-e660-4e2e-b81f-f6c7df63f443 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:58.952449 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a127dc-e660-4e2e-b81f-f6c7df63f443 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:58.954026 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a127dc-e660-4e2e-b81f-f6c7df63f443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:58.955609 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a127dc-e660-4e2e-b81f-f6c7df63f443 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:58.957564 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 18/38] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:23:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:23:58.958923 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3e993-29d4-43cd-851a-87dfa530185c None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:58.959220 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3e993-29d4-43cd-851a-87dfa530185c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:58.959590 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3e993-29d4-43cd-851a-87dfa530185c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:58.979610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-3ee3e993-29d4-43cd-851a-87dfa530185c None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:23:58.981397 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3ee3e993-29d4-43cd-851a-87dfa530185c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:23:58.983856 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:23:59.061827 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 19/39] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:23:59.063263 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5521e7-f3c5-44a5-9dec-a4318111001a None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:59.063559 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5521e7-f3c5-44a5-9dec-a4318111001a None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:59.063879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5521e7-f3c5-44a5-9dec-a4318111001a None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:59.065464 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 20/40] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:23:59.067875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5521e7-f3c5-44a5-9dec-a4318111001a None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:59.100149 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 21/41] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Jul 18 08:23:58 2022] POST /identity/v3/roles => generated 219 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:23:59.101792 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbe081-e940-436b-b671-5f1de0c3ca37 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:59.102098 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbe081-e940-436b-b671-5f1de0c3ca37 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:59.102386 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbe081-e940-436b-b671-5f1de0c3ca37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:59.126694 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-73dbe081-e940-436b-b671-5f1de0c3ca37 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:23:59.128392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-73dbe081-e940-436b-b671-5f1de0c3ca37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:23:59.234172 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5521e7-f3c5-44a5-9dec-a4318111001a None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:59.296392 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5779f-0d5b-4b7f-95b4-ffcaa51f0b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:59.297365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5779f-0d5b-4b7f-95b4-ffcaa51f0b70 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:59.297656 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5779f-0d5b-4b7f-95b4-ffcaa51f0b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:59.298238 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5779f-0d5b-4b7f-95b4-ffcaa51f0b70 None admin] PATH_INFO: `/v3/domains/default/users/93ad24fe28ec40dbbca364cbdb34972d/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:59.301995 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5779f-0d5b-4b7f-95b4-ffcaa51f0b70 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=93ad24fe28ec40dbbca364cbdb34972d, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:59.306517 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5779f-0d5b-4b7f-95b4-ffcaa51f0b70 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:59.368535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 22/42] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:23:59.369318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dbe081-e940-436b-b671-5f1de0c3ca37 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:59.398150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:23:59.404102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 21/43] 10.222.0.57 () {58 vars in 1361 bytes} [Mon Jul 18 08:23:59 2022] PUT /identity/v3/domains/default/users/93ad24fe28ec40dbbca364cbdb34972d/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:23:59.404958 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5779f-0d5b-4b7f-95b4-ffcaa51f0b70 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:59.440773 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:23:59.516613 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dbe081-e940-436b-b671-5f1de0c3ca37 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:59.542543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5779f-0d5b-4b7f-95b4-ffcaa51f0b70 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:59.563348 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a07586-6e0e-4007-ba4f-9e745d58a577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:59.564318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a07586-6e0e-4007-ba4f-9e745d58a577 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:59.564742 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a07586-6e0e-4007-ba4f-9e745d58a577 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:59.565152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a07586-6e0e-4007-ba4f-9e745d58a577 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:59.566125 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a07586-6e0e-4007-ba4f-9e745d58a577 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:59.568144 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a07586-6e0e-4007-ba4f-9e745d58a577 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:59.601531 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540421f-f9ad-403a-98c7-21cfdc55e17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:59.602501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540421f-f9ad-403a-98c7-21cfdc55e17e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:59.602875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540421f-f9ad-403a-98c7-21cfdc55e17e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:59.603324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540421f-f9ad-403a-98c7-21cfdc55e17e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:59.604735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540421f-f9ad-403a-98c7-21cfdc55e17e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:59.606416 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540421f-f9ad-403a-98c7-21cfdc55e17e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:59.608269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 22/44] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:23:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:23:59.616487 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540421f-f9ad-403a-98c7-21cfdc55e17e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:23:59.654394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 23/45] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Jul 18 08:23:59 2022] POST /identity/v3/roles => generated 223 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:23:59.734676 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:23:59.781957 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540421f-f9ad-403a-98c7-21cfdc55e17e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:23:59.824638 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302f339-b1e8-4211-bbb1-6c2fb9eb6006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:23:59.825754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302f339-b1e8-4211-bbb1-6c2fb9eb6006 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:23:59.826282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302f339-b1e8-4211-bbb1-6c2fb9eb6006 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:23:59.826769 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302f339-b1e8-4211-bbb1-6c2fb9eb6006 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:23:59.827802 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1302f339-b1e8-4211-bbb1-6c2fb9eb6006 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:23:59.829607 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1302f339-b1e8-4211-bbb1-6c2fb9eb6006 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:23:59.928730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 23/46] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 18 08:23:59 2022] POST /identity/v3/projects => generated 312 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:24:01.578129 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058e9e9-6d06-4c9d-bcd0-a822e2a6f2af None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:01.578917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058e9e9-6d06-4c9d-bcd0-a822e2a6f2af None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:01.579056 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058e9e9-6d06-4c9d-bcd0-a822e2a6f2af None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:01.580649 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 24/47] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:01.587516 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d2954-b789-4277-abe8-13e1280492f2 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:01.588085 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d2954-b789-4277-abe8-13e1280492f2 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:01.588567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d2954-b789-4277-abe8-13e1280492f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:01.610923 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-765d2954-b789-4277-abe8-13e1280492f2 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:01.612313 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-765d2954-b789-4277-abe8-13e1280492f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:01.692147 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 24/48] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:01.715984 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058e9e9-6d06-4c9d-bcd0-a822e2a6f2af None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:01.834737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058e9e9-6d06-4c9d-bcd0-a822e2a6f2af None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:01.840389 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:01.889811 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559d9ee-82fe-45db-bece-babb6f09c0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:01.890703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559d9ee-82fe-45db-bece-babb6f09c0cf None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:01.891035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559d9ee-82fe-45db-bece-babb6f09c0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:01.891355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559d9ee-82fe-45db-bece-babb6f09c0cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:01.903879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559d9ee-82fe-45db-bece-babb6f09c0cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:01.913039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-b559d9ee-82fe-45db-bece-babb6f09c0cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:24:01.913039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:01.913039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:01.913039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:01.913039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:01.913039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:01.913039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:01.913039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:01.913039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:01.913039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:01.913039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:01.913039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:01.913039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:01.913039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:01.913039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:01.913039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:01.914925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:01.914925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:01.914925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:01.914925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:01.914925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:01.914925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:01.914925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:01.914925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:01.914925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:01.914925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:01.914925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:01.914925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:01.914925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:01.914925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:01.914925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:01.914925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:01.916065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:01.916065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:01.916065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:01.916065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:01.916065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:01.916065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:01.916065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:01.916065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:01.916065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:01.916065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:01.916065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:24:01.916065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:01.916065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 25/49] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:24:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:01.925231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d2954-b789-4277-abe8-13e1280492f2 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:01.983182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:02.042664 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d2954-b789-4277-abe8-13e1280492f2 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:02.078068 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff39ed-d432-4bb9-9e87-653c788297da None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:02.078538 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff39ed-d432-4bb9-9e87-653c788297da None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:02.078995 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff39ed-d432-4bb9-9e87-653c788297da None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:02.081552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 26/50] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:24:02.089008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f13e2-a8d8-44ec-b7ef-0f67ee15f10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:02.090418 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f13e2-a8d8-44ec-b7ef-0f67ee15f10f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:02.090798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f13e2-a8d8-44ec-b7ef-0f67ee15f10f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:02.090969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c105f2-3f9f-4d11-a31b-80c5c2d594a8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:02.091239 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f13e2-a8d8-44ec-b7ef-0f67ee15f10f None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:02.091392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c105f2-3f9f-4d11-a31b-80c5c2d594a8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:02.091649 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c105f2-3f9f-4d11-a31b-80c5c2d594a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:02.092346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7f13e2-a8d8-44ec-b7ef-0f67ee15f10f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:02.092751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7f13e2-a8d8-44ec-b7ef-0f67ee15f10f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:02.094275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7f13e2-a8d8-44ec-b7ef-0f67ee15f10f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:02.108915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 25/51] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:24:02.111626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-03c105f2-3f9f-4d11-a31b-80c5c2d594a8 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:02.113011 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-03c105f2-3f9f-4d11-a31b-80c5c2d594a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:02.115227 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f13e2-a8d8-44ec-b7ef-0f67ee15f10f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:02.209178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 27/52] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:24:02.235352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c105f2-3f9f-4d11-a31b-80c5c2d594a8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:02.244678 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f13e2-a8d8-44ec-b7ef-0f67ee15f10f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:02.350261 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:02.358447 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c105f2-3f9f-4d11-a31b-80c5c2d594a8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:02.367449 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:02.403401 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa0cd3-2e4f-4d3d-b3e1-b2ee9c480777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:02.404673 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa0cd3-2e4f-4d3d-b3e1-b2ee9c480777 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:02.405046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa0cd3-2e4f-4d3d-b3e1-b2ee9c480777 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:02.405332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa0cd3-2e4f-4d3d-b3e1-b2ee9c480777 None admin] PATH_INFO: `/v3/users/93ad24fe28ec40dbbca364cbdb34972d` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:02.405456 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2711614-2990-4c2e-ae6f-3c18ae2963f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:02.406722 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa0cd3-2e4f-4d3d-b3e1-b2ee9c480777 None admin] RBAC: Authorizing `identity:get_user(user_id=93ad24fe28ec40dbbca364cbdb34972d)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:02.406856 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2711614-2990-4c2e-ae6f-3c18ae2963f8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:02.407341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2711614-2990-4c2e-ae6f-3c18ae2963f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:02.408070 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2711614-2990-4c2e-ae6f-3c18ae2963f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:02.408474 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa0cd3-2e4f-4d3d-b3e1-b2ee9c480777 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:02.410223 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 26/53] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:02 2022] GET /identity/v3/users/93ad24fe28ec40dbbca364cbdb34972d => generated 247 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:24:02.410368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2711614-2990-4c2e-ae6f-3c18ae2963f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:02.412755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2711614-2990-4c2e-ae6f-3c18ae2963f8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:02.415102 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 28/54] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:24:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:02.421031 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2711614-2990-4c2e-ae6f-3c18ae2963f8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:02.421449 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa0cd3-2e4f-4d3d-b3e1-b2ee9c480777 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:02.548282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2711614-2990-4c2e-ae6f-3c18ae2963f8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:02.562919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa0cd3-2e4f-4d3d-b3e1-b2ee9c480777 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:02.601173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e5d93-e3fe-418b-9e58-b493e8fcee6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:02.602218 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e5d93-e3fe-418b-9e58-b493e8fcee6a None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:02.602784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e5d93-e3fe-418b-9e58-b493e8fcee6a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:02.603296 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e5d93-e3fe-418b-9e58-b493e8fcee6a None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:02.604809 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e5d93-e3fe-418b-9e58-b493e8fcee6a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:02.605064 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a71ae-3264-4157-80ff-b27d4aa2eb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:02.606217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a71ae-3264-4157-80ff-b27d4aa2eb0a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:02.606511 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a71ae-3264-4157-80ff-b27d4aa2eb0a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:02.606794 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a71ae-3264-4157-80ff-b27d4aa2eb0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:02.607731 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e5d93-e3fe-418b-9e58-b493e8fcee6a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:02.608298 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a71ae-3264-4157-80ff-b27d4aa2eb0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:02.609943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a71ae-3264-4157-80ff-b27d4aa2eb0a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:02.611708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 27/55] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:24:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:02.618226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a71ae-3264-4157-80ff-b27d4aa2eb0a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:02.676797 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:02.741464 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a71ae-3264-4157-80ff-b27d4aa2eb0a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:02.743700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 29/56] 10.222.0.57 () {60 vars in 1140 bytes} [Mon Jul 18 08:24:02 2022] POST /identity/v3/projects => generated 298 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:24:02.798419 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a27b40-d08a-4ecd-a5b1-0f01a2abbb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:02.799601 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a27b40-d08a-4ecd-a5b1-0f01a2abbb6a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:02.800160 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a27b40-d08a-4ecd-a5b1-0f01a2abbb6a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:02.800658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a27b40-d08a-4ecd-a5b1-0f01a2abbb6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:02.802425 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a27b40-d08a-4ecd-a5b1-0f01a2abbb6a None admin] RBAC: Adding query filter params (role.id=467d49b8a0d3435aaba58464cc8ef85c, scope.domain.id=default, user.id=93ad24fe28ec40dbbca364cbdb34972d) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:02.802939 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a27b40-d08a-4ecd-a5b1-0f01a2abbb6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:02.805833 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a27b40-d08a-4ecd-a5b1-0f01a2abbb6a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:02.822456 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 28/57] 10.222.0.57 () {56 vars in 1322 bytes} [Mon Jul 18 08:24:02 2022] GET /identity/v3/role_assignments?user.id=93ad24fe28ec40dbbca364cbdb34972d&scope.domain.id=default&role.id=467d49b8a0d3435aaba58464cc8ef85c => generated 530 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:24:04.876772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8c9a6-ecf8-42c6-8fb7-74e9c3039840 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:04.877322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8c9a6-ecf8-42c6-8fb7-74e9c3039840 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:04.877419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8c9a6-ecf8-42c6-8fb7-74e9c3039840 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:04.878745 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 30/58] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:04.884005 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb53e15-560d-4f63-a084-61f394c174b6 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:04.884411 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb53e15-560d-4f63-a084-61f394c174b6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:04.884589 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb53e15-560d-4f63-a084-61f394c174b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:04.901789 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-1bb53e15-560d-4f63-a084-61f394c174b6 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:04.903570 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1bb53e15-560d-4f63-a084-61f394c174b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:04.969791 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 29/59] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:24:04.994178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8c9a6-ecf8-42c6-8fb7-74e9c3039840 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:05.010867 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:05.096101 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8c9a6-ecf8-42c6-8fb7-74e9c3039840 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:05.130860 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda0e66-6a86-4f66-9a22-bb7a14f111c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:05.131658 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda0e66-6a86-4f66-9a22-bb7a14f111c0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:05.132047 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda0e66-6a86-4f66-9a22-bb7a14f111c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:05.132330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda0e66-6a86-4f66-9a22-bb7a14f111c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:05.133530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda0e66-6a86-4f66-9a22-bb7a14f111c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:05.135010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda0e66-6a86-4f66-9a22-bb7a14f111c0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:05.137212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 31/60] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:24:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:05.143042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb53e15-560d-4f63-a084-61f394c174b6 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:05.183575 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:05.240860 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb53e15-560d-4f63-a084-61f394c174b6 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:05.274639 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76122bbd-e2e4-4c0e-8c67-bc3205376725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:05.275314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76122bbd-e2e4-4c0e-8c67-bc3205376725 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:05.275602 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76122bbd-e2e4-4c0e-8c67-bc3205376725 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:05.275911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76122bbd-e2e4-4c0e-8c67-bc3205376725 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:05.276638 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76122bbd-e2e4-4c0e-8c67-bc3205376725 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:05.278877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76122bbd-e2e4-4c0e-8c67-bc3205376725 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:05.398027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 30/61] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 18 08:24:05 2022] POST /identity/v3/users => generated 275 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:24:08.602078 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d379da-f4e5-4ef2-aa7c-ae2ea63a2a17 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:08.602754 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d379da-f4e5-4ef2-aa7c-ae2ea63a2a17 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:08.602754 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d379da-f4e5-4ef2-aa7c-ae2ea63a2a17 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:08.604218 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 32/62] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:08.610026 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d0f71-aff9-4261-b358-eedc29fe864d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:08.610490 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d0f71-aff9-4261-b358-eedc29fe864d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:08.610624 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d0f71-aff9-4261-b358-eedc29fe864d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:08.637273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-e17d0f71-aff9-4261-b358-eedc29fe864d None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:08.638490 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e17d0f71-aff9-4261-b358-eedc29fe864d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:08.726397 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 31/63] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:08.762986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d379da-f4e5-4ef2-aa7c-ae2ea63a2a17 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:08.896154 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:08.908064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d379da-f4e5-4ef2-aa7c-ae2ea63a2a17 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:08.955594 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c2699-9d5c-4736-a6d8-a5f650a70eec None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:08.955958 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c2699-9d5c-4736-a6d8-a5f650a70eec None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:08.956200 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c2699-9d5c-4736-a6d8-a5f650a70eec None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:08.957487 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5547d37-2442-405b-9695-b9b53fec4b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:08.957626 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 32/64] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:08.958364 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5547d37-2442-405b-9695-b9b53fec4b0a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:08.958658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb2eaf-5cde-45af-993a-e17b3896682e None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:08.958768 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5547d37-2442-405b-9695-b9b53fec4b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:08.958953 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5547d37-2442-405b-9695-b9b53fec4b0a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:08.959049 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb2eaf-5cde-45af-993a-e17b3896682e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:08.959213 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb2eaf-5cde-45af-993a-e17b3896682e None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:08.964744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 33/65] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:08.965871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf54bb-d827-42d0-be54-303a4ed787fb None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:08.966225 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf54bb-d827-42d0-be54-303a4ed787fb None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:08.966517 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf54bb-d827-42d0-be54-303a4ed787fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:08.972077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5547d37-2442-405b-9695-b9b53fec4b0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:08.980566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f5547d37-2442-405b-9695-b9b53fec4b0a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:08.980566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:08.980566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:08.980566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:08.980566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:08.980566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:08.980566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:08.980566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:08.980566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:08.980566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:08.980566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:08.980566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:08.980566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:08.980566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:08.980566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:08.980566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:08.981559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:08.981559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:08.981559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:08.981559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:08.981559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:08.981559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:08.981559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:08.981559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:08.981559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:08.981559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:08.981559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:08.981559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:08.981559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:08.981559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:08.981559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:08.981559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:08.982369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:08.982369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:08.982369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:08.982369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:08.982369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:08.982369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:08.982369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:08.982369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:08.982369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:08.982369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:08.982369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:08.982369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:08.982369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 33/66] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:08.983829 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce83f6-9ac9-44c1-9761-e30f553c51d0 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:08.983829 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce83f6-9ac9-44c1-9761-e30f553c51d0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:08.983829 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce83f6-9ac9-44c1-9761-e30f553c51d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:08.986816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-dbdf54bb-d827-42d0-be54-303a4ed787fb None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:08.988086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-dbdf54bb-d827-42d0-be54-303a4ed787fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:09.003144 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-08ce83f6-9ac9-44c1-9761-e30f553c51d0 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:09.004407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-08ce83f6-9ac9-44c1-9761-e30f553c51d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:09.024139 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:09.069863 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 34/67] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:24:09.070618 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf54bb-d827-42d0-be54-303a4ed787fb None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:09.098019 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 34/68] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:09.115979 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce83f6-9ac9-44c1-9761-e30f553c51d0 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:09.195225 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf54bb-d827-42d0-be54-303a4ed787fb None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:09.208478 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:09.235573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b58c1-9d7c-417f-a749-c538a7d2de70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:09.236327 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b58c1-9d7c-417f-a749-c538a7d2de70 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:09.236327 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b58c1-9d7c-417f-a749-c538a7d2de70 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:09.236882 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b58c1-9d7c-417f-a749-c538a7d2de70 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:09.237672 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7b58c1-9d7c-417f-a749-c538a7d2de70 None admin] RBAC: Adding query filter params (name=member) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:09.237981 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7b58c1-9d7c-417f-a749-c538a7d2de70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:09.239378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7b58c1-9d7c-417f-a749-c538a7d2de70 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:09.243785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce83f6-9ac9-44c1-9761-e30f553c51d0 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:09.248550 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 35/69] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:09.249562 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b58c1-9d7c-417f-a749-c538a7d2de70 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:09.272027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:09.284772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e882d4-4db9-4960-b566-f44017142b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:09.289479 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e882d4-4db9-4960-b566-f44017142b27 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:09.289479 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e882d4-4db9-4960-b566-f44017142b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:09.289479 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e882d4-4db9-4960-b566-f44017142b27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:09.298746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e882d4-4db9-4960-b566-f44017142b27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:09.306956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-10e882d4-4db9-4960-b566-f44017142b27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:24:09.306956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:09.306956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:09.306956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:09.306956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:09.306956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:09.306956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:09.306956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:09.306956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:09.306956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:09.306956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:09.306956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:09.306956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:09.306956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:09.306956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:09.306956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:09.308012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:09.308012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:09.308012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:09.308012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:09.308012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:09.308012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:09.308012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:09.308012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:09.308012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:09.308012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:09.308012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:09.308012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:09.308012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:09.308012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:09.308012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:09.308012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:09.308894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:09.308894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:09.308894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:09.308894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:09.308894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:09.308894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:09.308894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:09.308894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:09.308894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:09.308894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:09.308894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:24:09.308894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:09.310894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 35/70] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:24:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:09.313073 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5da9dc-3f2c-4f6f-9b3d-643cf62dab65 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:09.313367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5da9dc-3f2c-4f6f-9b3d-643cf62dab65 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:09.313652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5da9dc-3f2c-4f6f-9b3d-643cf62dab65 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:09.315144 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 36/71] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:09.315830 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5da9dc-3f2c-4f6f-9b3d-643cf62dab65 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:09.387135 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b58c1-9d7c-417f-a749-c538a7d2de70 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:09.426367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb9c8e-d8a2-4859-9d45-e29c0d2592c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:09.427191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb9c8e-d8a2-4859-9d45-e29c0d2592c2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:09.427550 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb9c8e-d8a2-4859-9d45-e29c0d2592c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:09.427911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb9c8e-d8a2-4859-9d45-e29c0d2592c2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:09.431088 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5da9dc-3f2c-4f6f-9b3d-643cf62dab65 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:09.441354 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb9c8e-d8a2-4859-9d45-e29c0d2592c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:09.450990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-c5cb9c8e-d8a2-4859-9d45-e29c0d2592c2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:24:09.450990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:09.450990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:09.450990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:09.450990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:09.450990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:09.450990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:09.450990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:09.450990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:09.450990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:09.450990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:09.450990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:09.450990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:09.450990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:09.450990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:09.450990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:09.452078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:09.452078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:09.452078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:09.452078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:09.452078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:09.452078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:09.452078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:09.452078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:09.452078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:09.452078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:09.452078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:09.452078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:09.452078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:09.452078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:09.452078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:09.452078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:09.452849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:09.452849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:09.452849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:09.452849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:09.452849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:09.452849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:09.452849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:09.452849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:09.452849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:09.452849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:09.452849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:24:09.452849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:24:09.452849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 36/72] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:24:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:09.453487 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb9c8e-d8a2-4859-9d45-e29c0d2592c2 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:09.465821 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:09.470862 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8a6a9-29b4-4281-b0c9-e22fa86db44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:09.471821 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8a6a9-29b4-4281-b0c9-e22fa86db44b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:09.478158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8a6a9-29b4-4281-b0c9-e22fa86db44b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:09.478158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8a6a9-29b4-4281-b0c9-e22fa86db44b None admin] PATH_INFO: `/v3/users/a9e3e8eb9c8c4db0ae0629919a5b1a56` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:09.487460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8a6a9-29b4-4281-b0c9-e22fa86db44b None admin] RBAC: Authorizing `identity:get_user(user_id=a9e3e8eb9c8c4db0ae0629919a5b1a56)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:09.489038 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8a6a9-29b4-4281-b0c9-e22fa86db44b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:09.490781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 37/73] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:09 2022] GET /identity/v3/users/a9e3e8eb9c8c4db0ae0629919a5b1a56 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:09.492172 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48f440-8e0a-4263-9001-1b6d180433ab None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:09.492473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48f440-8e0a-4263-9001-1b6d180433ab None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:09.493016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48f440-8e0a-4263-9001-1b6d180433ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:09.510570 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-cc48f440-8e0a-4263-9001-1b6d180433ab None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:09.512008 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cc48f440-8e0a-4263-9001-1b6d180433ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:09.537319 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:09.561560 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb9c8e-d8a2-4859-9d45-e29c0d2592c2 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:09.581487 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 38/74] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:09.583077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd8b15-66ba-4db7-a5af-6744fd12b12d None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:09.583451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd8b15-66ba-4db7-a5af-6744fd12b12d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:09.583767 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd8b15-66ba-4db7-a5af-6744fd12b12d None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:09.585265 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 39/75] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:09.585901 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd8b15-66ba-4db7-a5af-6744fd12b12d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:09.606606 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e8e8e-1e88-479e-b837-73020e859206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:09.607306 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e8e8e-1e88-479e-b837-73020e859206 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:09.607601 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e8e8e-1e88-479e-b837-73020e859206 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:09.607963 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e8e8e-1e88-479e-b837-73020e859206 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:09.608777 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e8e8e-1e88-479e-b837-73020e859206 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:09.609094 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e8e8e-1e88-479e-b837-73020e859206 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:09.610562 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e8e8e-1e88-479e-b837-73020e859206 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:09.621340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 37/76] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:09.622249 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e8e8e-1e88-479e-b837-73020e859206 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:09.701564 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd8b15-66ba-4db7-a5af-6744fd12b12d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:09.722783 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:09.732274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e8e8e-1e88-479e-b837-73020e859206 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:09.737979 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74446483-64b3-4dcb-b55e-c0757941929d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:09.738658 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74446483-64b3-4dcb-b55e-c0757941929d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:09.738945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74446483-64b3-4dcb-b55e-c0757941929d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:09.739225 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74446483-64b3-4dcb-b55e-c0757941929d None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:09.740026 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74446483-64b3-4dcb-b55e-c0757941929d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:09.740743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74446483-64b3-4dcb-b55e-c0757941929d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:09.741746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74446483-64b3-4dcb-b55e-c0757941929d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:09.752067 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 40/77] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:24:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:09.753350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d7640-8f8d-4ec0-9a33-7fd86b6790aa None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:09.753639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d7640-8f8d-4ec0-9a33-7fd86b6790aa None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:09.753919 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d7640-8f8d-4ec0-9a33-7fd86b6790aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:09.770809 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-9d5d7640-8f8d-4ec0-9a33-7fd86b6790aa None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:09.772055 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9d5d7640-8f8d-4ec0-9a33-7fd86b6790aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:09.773047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae11f2a-e822-4434-91b6-d26c791747ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:09.773708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae11f2a-e822-4434-91b6-d26c791747ff None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:09.773997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae11f2a-e822-4434-91b6-d26c791747ff None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:09.774274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae11f2a-e822-4434-91b6-d26c791747ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:09.786960 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:09.851397 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae11f2a-e822-4434-91b6-d26c791747ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:09.852030 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 41/78] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:09.852582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d7640-8f8d-4ec0-9a33-7fd86b6790aa None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:09.852934 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae11f2a-e822-4434-91b6-d26c791747ff None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:09.981296 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 38/79] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:24:09.984567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae11f2a-e822-4434-91b6-d26c791747ff None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:09.986151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d7640-8f8d-4ec0-9a33-7fd86b6790aa None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:10.001030 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:10.023054 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667eb6a8-bdbb-4438-b2aa-d68f1058d426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:10.024395 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667eb6a8-bdbb-4438-b2aa-d68f1058d426 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:10.024395 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667eb6a8-bdbb-4438-b2aa-d68f1058d426 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:10.024600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667eb6a8-bdbb-4438-b2aa-d68f1058d426 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:10.025744 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667eb6a8-bdbb-4438-b2aa-d68f1058d426 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:10.027795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667eb6a8-bdbb-4438-b2aa-d68f1058d426 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:10.029463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 42/80] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:24:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:10.029967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:10.030251 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667eb6a8-bdbb-4438-b2aa-d68f1058d426 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:10.097982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae11f2a-e822-4434-91b6-d26c791747ff None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:10.137600 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe23856-f7cb-4be1-8e0b-1e9736132695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:10.138183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe23856-f7cb-4be1-8e0b-1e9736132695 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:10.138472 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe23856-f7cb-4be1-8e0b-1e9736132695 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:10.138756 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe23856-f7cb-4be1-8e0b-1e9736132695 None admin] PATH_INFO: `/v3/users/93ad24fe28ec40dbbca364cbdb34972d` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:10.140071 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe23856-f7cb-4be1-8e0b-1e9736132695 None admin] RBAC: Authorizing `identity:get_user(user_id=93ad24fe28ec40dbbca364cbdb34972d)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:10.141531 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe23856-f7cb-4be1-8e0b-1e9736132695 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:10.143203 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 39/81] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:09 2022] GET /identity/v3/users/93ad24fe28ec40dbbca364cbdb34972d => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:10.144011 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe23856-f7cb-4be1-8e0b-1e9736132695 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:10.144732 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667eb6a8-bdbb-4438-b2aa-d68f1058d426 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:10.184115 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c9281-78ff-4821-94fe-42aceaea3407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:10.185404 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c9281-78ff-4821-94fe-42aceaea3407 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:10.185880 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c9281-78ff-4821-94fe-42aceaea3407 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:10.186337 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c9281-78ff-4821-94fe-42aceaea3407 None admin] PATH_INFO: `/v3/users/a9e3e8eb9c8c4db0ae0629919a5b1a56` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:10.189881 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c9281-78ff-4821-94fe-42aceaea3407 None admin] RBAC: Authorizing `identity:get_user(user_id=a9e3e8eb9c8c4db0ae0629919a5b1a56)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:10.191401 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c9281-78ff-4821-94fe-42aceaea3407 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:10.193467 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 43/82] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:10 2022] GET /identity/v3/users/a9e3e8eb9c8c4db0ae0629919a5b1a56 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:10.194071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c9281-78ff-4821-94fe-42aceaea3407 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:10.248455 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:10.265391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe23856-f7cb-4be1-8e0b-1e9736132695 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:10.275276 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:10.303999 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fa683-8a76-4cde-87bc-a6fd6d1ffc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:10.304812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fa683-8a76-4cde-87bc-a6fd6d1ffc4f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:10.305107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fa683-8a76-4cde-87bc-a6fd6d1ffc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:10.305392 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fa683-8a76-4cde-87bc-a6fd6d1ffc4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:10.308051 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c9281-78ff-4821-94fe-42aceaea3407 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:10.316864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fa683-8a76-4cde-87bc-a6fd6d1ffc4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:10.325896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-fc8fa683-8a76-4cde-87bc-a6fd6d1ffc4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:10.325896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:10.325896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:10.325896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:10.325896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:10.325896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:10.325896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:10.325896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:10.325896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:10.325896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:10.325896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:10.325896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:10.325896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:10.325896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:10.325896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:10.325896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:10.326985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:10.326985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:10.326985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:10.326985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:10.326985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:10.326985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:10.326985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:10.326985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:10.326985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:10.326985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:10.326985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:10.326985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:10.326985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:10.326985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:10.326985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:10.326985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:10.327980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:10.327980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:10.327980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:10.327980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:10.327980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:10.327980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:10.327980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:10.327980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:10.327980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:10.327980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:10.327980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:10.327980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:24:10.327980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 40/83] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:10.328774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fa683-8a76-4cde-87bc-a6fd6d1ffc4f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:10.354488 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec3541-0497-4e7f-90af-ee15b878d856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:10.355432 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec3541-0497-4e7f-90af-ee15b878d856 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:10.355784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec3541-0497-4e7f-90af-ee15b878d856 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:10.356066 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec3541-0497-4e7f-90af-ee15b878d856 None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:10.357316 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec3541-0497-4e7f-90af-ee15b878d856 None admin] RBAC: Authorizing `identity:get_project(project_id=791daf53d0364b258d9cfb153ffa3111)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:10.358814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec3541-0497-4e7f-90af-ee15b878d856 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:10.360583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 44/84] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:10 2022] GET /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:10.361359 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec3541-0497-4e7f-90af-ee15b878d856 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:10.450087 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fa683-8a76-4cde-87bc-a6fd6d1ffc4f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:10.475927 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec3541-0497-4e7f-90af-ee15b878d856 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:10.489533 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3392bb3-e356-4b4c-a25b-e93c61ef6939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:10.490216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3392bb3-e356-4b4c-a25b-e93c61ef6939 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:10.490507 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3392bb3-e356-4b4c-a25b-e93c61ef6939 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:10.490795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3392bb3-e356-4b4c-a25b-e93c61ef6939 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:10.491558 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3392bb3-e356-4b4c-a25b-e93c61ef6939 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:10.492872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:10.494154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3392bb3-e356-4b4c-a25b-e93c61ef6939 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:10.514792 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b665352-0cd2-4c0b-b14b-ffd0cee0763e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:10.515373 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b665352-0cd2-4c0b-b14b-ffd0cee0763e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:10.515694 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b665352-0cd2-4c0b-b14b-ffd0cee0763e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:10.515971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b665352-0cd2-4c0b-b14b-ffd0cee0763e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:10.589465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b665352-0cd2-4c0b-b14b-ffd0cee0763e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:10.590981 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b665352-0cd2-4c0b-b14b-ffd0cee0763e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:10.594665 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 41/85] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 18 08:24:10 2022] POST /identity/v3/users => generated 289 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:24:10.600397 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3392bb3-e356-4b4c-a25b-e93c61ef6939 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:10.639552 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:10.748023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3392bb3-e356-4b4c-a25b-e93c61ef6939 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:10.751781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 45/86] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:24:10.752638 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b665352-0cd2-4c0b-b14b-ffd0cee0763e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:10.777452 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:10.823294 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287cb31-b521-4fd6-aea3-b050e90cee5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:10.823294 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287cb31-b521-4fd6-aea3-b050e90cee5f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:10.823294 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287cb31-b521-4fd6-aea3-b050e90cee5f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:10.823294 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287cb31-b521-4fd6-aea3-b050e90cee5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:10.917638 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b665352-0cd2-4c0b-b14b-ffd0cee0763e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:10.974944 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287cb31-b521-4fd6-aea3-b050e90cee5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:10.976667 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287cb31-b521-4fd6-aea3-b050e90cee5f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:10.985706 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118dc17b-2b62-46be-911c-f784fa9f1ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:10.986827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dc17b-2b62-46be-911c-f784fa9f1ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:10.988170 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dc17b-2b62-46be-911c-f784fa9f1ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:10.988652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dc17b-2b62-46be-911c-f784fa9f1ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:10.990639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118dc17b-2b62-46be-911c-f784fa9f1ff6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:10.991205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118dc17b-2b62-46be-911c-f784fa9f1ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:10.995167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118dc17b-2b62-46be-911c-f784fa9f1ff6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:11.012901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:11.015620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 46/87] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:11.016847 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118dc17b-2b62-46be-911c-f784fa9f1ff6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:11.118391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 42/88] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:24:11.119738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287cb31-b521-4fd6-aea3-b050e90cee5f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:11.170014 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:11.211512 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118dc17b-2b62-46be-911c-f784fa9f1ff6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:11.252428 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287cb31-b521-4fd6-aea3-b050e90cee5f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:11.278279 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e789972-5bfd-403f-a4bb-2825fa89f6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:11.279746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e789972-5bfd-403f-a4bb-2825fa89f6ac None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:11.280434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e789972-5bfd-403f-a4bb-2825fa89f6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:11.281145 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e789972-5bfd-403f-a4bb-2825fa89f6ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:11.283062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:11.283277 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e789972-5bfd-403f-a4bb-2825fa89f6ac None admin] RBAC: Adding query filter params (role.id=587af95739954600bbf2411974601f74, scope.project.id=791daf53d0364b258d9cfb153ffa3111, user.id=a9e3e8eb9c8c4db0ae0629919a5b1a56) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:11.284067 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e789972-5bfd-403f-a4bb-2825fa89f6ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:11.286999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e789972-5bfd-403f-a4bb-2825fa89f6ac None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:11.296182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b0701-fdd1-418d-a100-35af4898208a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:11.297064 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b0701-fdd1-418d-a100-35af4898208a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:11.307851 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b0701-fdd1-418d-a100-35af4898208a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:11.307851 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b0701-fdd1-418d-a100-35af4898208a None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:11.307851 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b0701-fdd1-418d-a100-35af4898208a None admin] RBAC: Authorizing `identity:get_project(project_id=791daf53d0364b258d9cfb153ffa3111)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:11.308377 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 47/89] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:24:11 2022] GET /identity/v3/role_assignments?user.id=a9e3e8eb9c8c4db0ae0629919a5b1a56&scope.project.id=791daf53d0364b258d9cfb153ffa3111&role.id=587af95739954600bbf2411974601f74 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:11.308538 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b0701-fdd1-418d-a100-35af4898208a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:11.308680 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e789972-5bfd-403f-a4bb-2825fa89f6ac None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:11.308811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 43/90] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:11 2022] GET /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:11.308811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b0701-fdd1-418d-a100-35af4898208a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:11.461960 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e789972-5bfd-403f-a4bb-2825fa89f6ac None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:11.476322 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b0701-fdd1-418d-a100-35af4898208a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:11.515812 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7a427-3aef-4de8-92ee-cd36ee8c1fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:11.523488 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7a427-3aef-4de8-92ee-cd36ee8c1fde None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:11.523488 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7a427-3aef-4de8-92ee-cd36ee8c1fde None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:11.523488 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7a427-3aef-4de8-92ee-cd36ee8c1fde None admin] PATH_INFO: `/v3/users/a9e3e8eb9c8c4db0ae0629919a5b1a56` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:11.525267 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7a427-3aef-4de8-92ee-cd36ee8c1fde None admin] RBAC: Authorizing `identity:get_user(user_id=a9e3e8eb9c8c4db0ae0629919a5b1a56)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:11.528524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7a427-3aef-4de8-92ee-cd36ee8c1fde None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:11.533571 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 48/91] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:11 2022] GET /identity/v3/users/a9e3e8eb9c8c4db0ae0629919a5b1a56 => generated 275 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:11.538413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7a427-3aef-4de8-92ee-cd36ee8c1fde None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:11.557740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b69463-0885-42df-afc5-18e97520b7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:11.559312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:11.559997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b69463-0885-42df-afc5-18e97520b7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:11.560293 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b69463-0885-42df-afc5-18e97520b7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:11.560581 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b69463-0885-42df-afc5-18e97520b7c3 None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:11.567070 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b69463-0885-42df-afc5-18e97520b7c3 None admin] RBAC: Authorizing `identity:get_project(project_id=791daf53d0364b258d9cfb153ffa3111)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:11.568617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b69463-0885-42df-afc5-18e97520b7c3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:11.570588 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 44/92] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:11 2022] GET /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:11.571405 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b69463-0885-42df-afc5-18e97520b7c3 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:11.705760 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:11.734445 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b69463-0885-42df-afc5-18e97520b7c3 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:11.774982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7157b5-3e3f-4755-aa4c-2d2adf8a9ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:11.775652 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7157b5-3e3f-4755-aa4c-2d2adf8a9ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:11.775973 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7157b5-3e3f-4755-aa4c-2d2adf8a9ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:11.776311 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7157b5-3e3f-4755-aa4c-2d2adf8a9ee5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:11.794249 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7a427-3aef-4de8-92ee-cd36ee8c1fde None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:11.869197 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7157b5-3e3f-4755-aa4c-2d2adf8a9ee5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:11.871924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7157b5-3e3f-4755-aa4c-2d2adf8a9ee5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:11.872576 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06cff1-61c4-4e0b-bec0-ba46c806ad8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:11.873270 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06cff1-61c4-4e0b-bec0-ba46c806ad8c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:11.873564 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06cff1-61c4-4e0b-bec0-ba46c806ad8c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:11.873839 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06cff1-61c4-4e0b-bec0-ba46c806ad8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:11.874709 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06cff1-61c4-4e0b-bec0-ba46c806ad8c None admin] RBAC: Adding query filter params (role.id=467d49b8a0d3435aaba58464cc8ef85c, scope.project.id=791daf53d0364b258d9cfb153ffa3111, user.id=93ad24fe28ec40dbbca364cbdb34972d) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:11.875031 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06cff1-61c4-4e0b-bec0-ba46c806ad8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:11.876547 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06cff1-61c4-4e0b-bec0-ba46c806ad8c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:11.890114 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 49/93] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:24:11 2022] GET /identity/v3/role_assignments?user.id=93ad24fe28ec40dbbca364cbdb34972d&scope.project.id=791daf53d0364b258d9cfb153ffa3111&role.id=467d49b8a0d3435aaba58464cc8ef85c => generated 259 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:11.891015 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06cff1-61c4-4e0b-bec0-ba46c806ad8c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:11.968328 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:12.035514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 45/94] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:24:12.041439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7157b5-3e3f-4755-aa4c-2d2adf8a9ee5 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:12.055901 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:12.100636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06cff1-61c4-4e0b-bec0-ba46c806ad8c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:12.163385 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7157b5-3e3f-4755-aa4c-2d2adf8a9ee5 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:12.173095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22610f90-8eef-4bab-9194-c849328f9bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:12.173792 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22610f90-8eef-4bab-9194-c849328f9bde None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:12.174089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22610f90-8eef-4bab-9194-c849328f9bde None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:12.174377 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22610f90-8eef-4bab-9194-c849328f9bde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:12.175256 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22610f90-8eef-4bab-9194-c849328f9bde None admin] RBAC: Adding query filter params (role.id=d2ba35e60f974010a5e7dd5c4bebecf8, scope.project.id=791daf53d0364b258d9cfb153ffa3111, user.id=a9e3e8eb9c8c4db0ae0629919a5b1a56) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:12.175582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22610f90-8eef-4bab-9194-c849328f9bde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:12.177140 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22610f90-8eef-4bab-9194-c849328f9bde None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:12.189925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 50/95] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:24:11 2022] GET /identity/v3/role_assignments?user.id=a9e3e8eb9c8c4db0ae0629919a5b1a56&scope.project.id=791daf53d0364b258d9cfb153ffa3111&role.id=d2ba35e60f974010a5e7dd5c4bebecf8 => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:12.204814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836d240-5b01-4809-b316-3639e4068c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:12.207056 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836d240-5b01-4809-b316-3639e4068c63 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:12.207556 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836d240-5b01-4809-b316-3639e4068c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:12.208175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836d240-5b01-4809-b316-3639e4068c63 None admin] PATH_INFO: `/v3/projects/4c4224e757934016a179499c9c15cf75` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:12.210272 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836d240-5b01-4809-b316-3639e4068c63 None admin] RBAC: Authorizing `identity:get_project(project_id=4c4224e757934016a179499c9c15cf75)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:12.213549 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836d240-5b01-4809-b316-3639e4068c63 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:12.217753 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 46/96] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:12 2022] GET /identity/v3/projects/4c4224e757934016a179499c9c15cf75 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:24:12.227091 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22610f90-8eef-4bab-9194-c849328f9bde None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:12.463239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22610f90-8eef-4bab-9194-c849328f9bde None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:12.540089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8681a-419b-4c31-8853-470ec4492225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:12.540555 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8681a-419b-4c31-8853-470ec4492225 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:12.540804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8681a-419b-4c31-8853-470ec4492225 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:12.543696 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8681a-419b-4c31-8853-470ec4492225 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:12.543696 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8681a-419b-4c31-8853-470ec4492225 None admin] RBAC: Adding query filter params (role.id=587af95739954600bbf2411974601f74, scope.project.id=4c4224e757934016a179499c9c15cf75, user.id=a9e3e8eb9c8c4db0ae0629919a5b1a56) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:12.543696 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8681a-419b-4c31-8853-470ec4492225 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:12.545370 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8681a-419b-4c31-8853-470ec4492225 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:12.551193 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 51/97] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:24:12 2022] GET /identity/v3/role_assignments?user.id=a9e3e8eb9c8c4db0ae0629919a5b1a56&scope.project.id=4c4224e757934016a179499c9c15cf75&role.id=587af95739954600bbf2411974601f74 => generated 259 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:14.008869 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee20d9-fe6b-4d0e-aa28-a1751f2ba2c2 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:14.009897 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee20d9-fe6b-4d0e-aa28-a1751f2ba2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:14.010379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee20d9-fe6b-4d0e-aa28-a1751f2ba2c2 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:14.013063 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 47/98] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:14.019088 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e060e-0247-468e-b08b-574c9a33c25f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:14.020071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e060e-0247-468e-b08b-574c9a33c25f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:14.020548 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e060e-0247-468e-b08b-574c9a33c25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:14.026195 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:14.041477 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c02e060e-0247-468e-b08b-574c9a33c25f None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:14.044280 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c02e060e-0247-468e-b08b-574c9a33c25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:14.145652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 52/99] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:14.180955 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee20d9-fe6b-4d0e-aa28-a1751f2ba2c2 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:14.213938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:14.306502 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee20d9-fe6b-4d0e-aa28-a1751f2ba2c2 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:14.348681 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8202d01-8b50-473e-9bda-6e3dd4f3ad1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:14.349796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8202d01-8b50-473e-9bda-6e3dd4f3ad1e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:14.350282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8202d01-8b50-473e-9bda-6e3dd4f3ad1e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:14.350740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8202d01-8b50-473e-9bda-6e3dd4f3ad1e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:14.369319 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8202d01-8b50-473e-9bda-6e3dd4f3ad1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:14.386962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-a8202d01-8b50-473e-9bda-6e3dd4f3ad1e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 08:24:14.386962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:14.386962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:14.386962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:14.386962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:14.386962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:14.386962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:14.386962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:14.386962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:14.386962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:14.386962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:14.386962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:14.386962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:14.386962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:14.386962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:14.386962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:14.388610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:14.388610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:14.388610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:14.388610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:14.388610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:14.388610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:14.388610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:14.388610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:14.388610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:14.388610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:14.388610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:14.388610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:14.388610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:14.388610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:14.388610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:14.388610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:14.390003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:14.390003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:14.390003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:14.390003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:14.390003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:14.390003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:14.390003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:14.390003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:14.390003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:14.390003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:14.390003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 08:24:14.390003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:24:14.393681 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 48/100] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:14.402972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e060e-0247-468e-b08b-574c9a33c25f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:14.583568 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e060e-0247-468e-b08b-574c9a33c25f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:14.663949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:14.679158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30391a49-9ccc-4767-b2dc-602923531722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:14.680325 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30391a49-9ccc-4767-b2dc-602923531722 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:14.680823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30391a49-9ccc-4767-b2dc-602923531722 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:14.681319 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30391a49-9ccc-4767-b2dc-602923531722 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:14.682854 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30391a49-9ccc-4767-b2dc-602923531722 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:14.687613 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30391a49-9ccc-4767-b2dc-602923531722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:14.687613 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30391a49-9ccc-4767-b2dc-602923531722 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:14.707196 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 53/101] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:14.713050 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8202d01-8b50-473e-9bda-6e3dd4f3ad1e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:14.872153 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:14.881826 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8202d01-8b50-473e-9bda-6e3dd4f3ad1e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:14.925391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b014161c-ef87-4a96-b94b-d1d9d6c81a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:14.926708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014161c-ef87-4a96-b94b-d1d9d6c81a6f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:14.927187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014161c-ef87-4a96-b94b-d1d9d6c81a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:14.927761 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014161c-ef87-4a96-b94b-d1d9d6c81a6f None admin] PATH_INFO: `/v3/users/ab00423871a34113bf9e5853f83e3011` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:14.946813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b014161c-ef87-4a96-b94b-d1d9d6c81a6f None admin] RBAC: Authorizing `identity:get_user(user_id=ab00423871a34113bf9e5853f83e3011)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:14.949264 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b014161c-ef87-4a96-b94b-d1d9d6c81a6f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:14.952019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 49/102] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:14 2022] GET /identity/v3/users/ab00423871a34113bf9e5853f83e3011 => generated 289 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:14.959834 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30391a49-9ccc-4767-b2dc-602923531722 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:15.115883 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30391a49-9ccc-4767-b2dc-602923531722 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:15.163973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1519d5-fcb5-4be6-bd19-d6037717d214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:15.164651 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1519d5-fcb5-4be6-bd19-d6037717d214 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:15.164944 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1519d5-fcb5-4be6-bd19-d6037717d214 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:15.165229 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1519d5-fcb5-4be6-bd19-d6037717d214 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:15.188412 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100c5ad-c158-40ab-941a-93d409df6ff6 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:15.189428 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100c5ad-c158-40ab-941a-93d409df6ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:15.189908 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100c5ad-c158-40ab-941a-93d409df6ff6 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:15.191799 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 50/103] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:15.197673 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73404042-949e-46bc-aa28-15c9ddee6d7e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:15.198387 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73404042-949e-46bc-aa28-15c9ddee6d7e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:15.198934 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73404042-949e-46bc-aa28-15c9ddee6d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:15.229097 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-73404042-949e-46bc-aa28-15c9ddee6d7e None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:15.234956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-73404042-949e-46bc-aa28-15c9ddee6d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:15.325732 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:15.344582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 51/104] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:24:15.348626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1519d5-fcb5-4be6-bd19-d6037717d214 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:15.350153 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1519d5-fcb5-4be6-bd19-d6037717d214 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:15.408681 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73404042-949e-46bc-aa28-15c9ddee6d7e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:15.557123 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 54/105] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:24:15.566077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1519d5-fcb5-4be6-bd19-d6037717d214 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:15.611441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:15.639571 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73404042-949e-46bc-aa28-15c9ddee6d7e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:15.690611 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b587202-5d61-410b-a9fd-1d4ff144b2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:15.691445 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b587202-5d61-410b-a9fd-1d4ff144b2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:15.691977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b587202-5d61-410b-a9fd-1d4ff144b2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:15.692267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b587202-5d61-410b-a9fd-1d4ff144b2c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:15.694800 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b587202-5d61-410b-a9fd-1d4ff144b2c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:15.696333 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b587202-5d61-410b-a9fd-1d4ff144b2c6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:15.698096 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 52/106] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:24:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:24:15.705255 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b587202-5d61-410b-a9fd-1d4ff144b2c6 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:15.776089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:15.798144 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1519d5-fcb5-4be6-bd19-d6037717d214 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:15.883185 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b8b02-0339-4119-9a49-27ed45c2c3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:15.884483 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b8b02-0339-4119-9a49-27ed45c2c3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:15.884975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b8b02-0339-4119-9a49-27ed45c2c3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:15.885472 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b8b02-0339-4119-9a49-27ed45c2c3a8 None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:15.890250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b8b02-0339-4119-9a49-27ed45c2c3a8 None admin] RBAC: Authorizing `identity:get_project(project_id=791daf53d0364b258d9cfb153ffa3111)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:15.892188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b8b02-0339-4119-9a49-27ed45c2c3a8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:15.894282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 55/107] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:15 2022] GET /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111 => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:24:15.900514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed5e98-4463-4f50-b796-cedd2ce413f2 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:15.901161 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed5e98-4463-4f50-b796-cedd2ce413f2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:15.902233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed5e98-4463-4f50-b796-cedd2ce413f2 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:15.904407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 56/108] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:24:15.908787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed5e98-4463-4f50-b796-cedd2ce413f2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:15.952502 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b587202-5d61-410b-a9fd-1d4ff144b2c6 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:16.014079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee60849-7586-4873-ab07-6fd15eb4516a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:16.016632 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee60849-7586-4873-ab07-6fd15eb4516a None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:16.017335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee60849-7586-4873-ab07-6fd15eb4516a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:16.018038 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee60849-7586-4873-ab07-6fd15eb4516a None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:16.020003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee60849-7586-4873-ab07-6fd15eb4516a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:16.024982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee60849-7586-4873-ab07-6fd15eb4516a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:16.054418 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:16.070100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed5e98-4463-4f50-b796-cedd2ce413f2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:16.139538 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7715d3-9329-454f-9a80-ab66be4bb31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:16.140636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7715d3-9329-454f-9a80-ab66be4bb31b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:16.141142 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7715d3-9329-454f-9a80-ab66be4bb31b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:16.141649 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7715d3-9329-454f-9a80-ab66be4bb31b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:16.143342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7715d3-9329-454f-9a80-ab66be4bb31b None admin] RBAC: Adding query filter params (role.id=24179486bd6a4eb985a83d47ca5c2b4f, scope.project.id=791daf53d0364b258d9cfb153ffa3111, user.id=ab00423871a34113bf9e5853f83e3011) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:16.143941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7715d3-9329-454f-9a80-ab66be4bb31b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:16.146771 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7715d3-9329-454f-9a80-ab66be4bb31b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:16.152107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 53/109] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 18 08:24:15 2022] POST /identity/v3/projects => generated 302 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:24:16.152107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2f803-df9a-4364-b14a-ddea1987f9aa None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:16.152107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2f803-df9a-4364-b14a-ddea1987f9aa None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:16.152107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2f803-df9a-4364-b14a-ddea1987f9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:16.160049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 57/110] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:24:15 2022] GET /identity/v3/role_assignments?user.id=ab00423871a34113bf9e5853f83e3011&scope.project.id=791daf53d0364b258d9cfb153ffa3111&role.id=24179486bd6a4eb985a83d47ca5c2b4f => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:16.162359 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef6360-f95b-49f2-9b6f-ae77e63528b4 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:16.162874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef6360-f95b-49f2-9b6f-ae77e63528b4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:16.163391 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef6360-f95b-49f2-9b6f-ae77e63528b4 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:16.166164 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 58/111] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:16.176025 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8caf084-c242-4463-ad10-b1e0b2940b90 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:16.176504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8caf084-c242-4463-ad10-b1e0b2940b90 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:16.177736 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8caf084-c242-4463-ad10-b1e0b2940b90 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:16.180053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 59/112] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:16 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:16.182005 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9efba6-1b52-4d92-8c78-74123a04e5fe None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:16.182501 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9efba6-1b52-4d92-8c78-74123a04e5fe None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:16.183557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9efba6-1b52-4d92-8c78-74123a04e5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:16.197808 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-5fd2f803-df9a-4364-b14a-ddea1987f9aa None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:16.202170 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5fd2f803-df9a-4364-b14a-ddea1987f9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:16.241702 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0f9efba6-1b52-4d92-8c78-74123a04e5fe None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:16.253491 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0f9efba6-1b52-4d92-8c78-74123a04e5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:16.330547 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:16.400783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 54/113] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:16.403068 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08deabe6-cf7a-4664-bd2f-b50858f4da4d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:16.403574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08deabe6-cf7a-4664-bd2f-b50858f4da4d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:16.404066 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08deabe6-cf7a-4664-bd2f-b50858f4da4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:16.424774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-08deabe6-cf7a-4664-bd2f-b50858f4da4d None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:16.425722 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-08deabe6-cf7a-4664-bd2f-b50858f4da4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:16.433790 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 60/114] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:16.466413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9efba6-1b52-4d92-8c78-74123a04e5fe None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:16.556453 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 55/115] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:16.557632 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08deabe6-cf7a-4664-bd2f-b50858f4da4d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:16.686473 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:16.724094 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9efba6-1b52-4d92-8c78-74123a04e5fe None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:16.783481 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08deabe6-cf7a-4664-bd2f-b50858f4da4d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:16.795261 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a3fca-2616-4cd6-9673-d7ca997b52b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:16.796677 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a3fca-2616-4cd6-9673-d7ca997b52b0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:16.797174 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a3fca-2616-4cd6-9673-d7ca997b52b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:16.797671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a3fca-2616-4cd6-9673-d7ca997b52b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:16.819653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a3fca-2616-4cd6-9673-d7ca997b52b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:16.836035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f36a3fca-2616-4cd6-9673-d7ca997b52b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:16.836035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:16.836035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:16.836035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:16.836035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:16.836035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:16.836035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:16.836035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:16.836035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:16.836035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:16.836035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:16.836035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:16.836035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:16.836035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:16.836035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:16.836035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:16.837948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:16.837948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:16.837948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:16.837948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:16.837948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:16.837948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:16.837948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:16.837948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:16.837948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:16.837948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:16.837948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:16.837948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:16.837948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:16.837948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:16.837948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:16.837948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:16.840184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:16.840184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:16.840184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:16.840184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:16.840184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:16.840184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:16.840184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:16.840184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:16.840184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:16.840184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:16.840184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:16.840184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:16.840184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 61/116] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:16.846240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1b5f6-146f-49eb-a27e-832b60bd765a None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:16.846240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1b5f6-146f-49eb-a27e-832b60bd765a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:16.846240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1b5f6-146f-49eb-a27e-832b60bd765a None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:16.858056 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 62/117] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:16.858056 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1b5f6-146f-49eb-a27e-832b60bd765a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:16.869688 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef87c83-6913-44ac-b5d1-04a149d402b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:16.872024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef87c83-6913-44ac-b5d1-04a149d402b1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:16.872824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef87c83-6913-44ac-b5d1-04a149d402b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:16.873591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef87c83-6913-44ac-b5d1-04a149d402b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:16.898198 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef87c83-6913-44ac-b5d1-04a149d402b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:16.907773 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:16.912984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-eef87c83-6913-44ac-b5d1-04a149d402b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:24:16.912984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:16.912984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:16.912984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:16.912984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:16.912984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:16.912984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:16.912984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:16.912984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:16.912984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:16.912984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:16.912984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:16.912984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:16.912984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:16.912984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:16.912984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:16.918373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:16.918373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:16.918373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:16.918373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:16.918373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:16.918373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:16.918373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:16.918373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:16.918373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:16.918373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:16.918373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:16.918373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:16.918373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:16.918373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:16.918373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:16.918373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:16.923816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:16.923816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:16.923816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:16.923816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:16.923816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:16.923816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:16.923816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:16.923816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:16.923816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:16.923816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:16.923816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:24:16.923816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:24:16.923816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 56/118] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:24:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:16.923816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda204d-1187-4e34-b8c2-32abae0da076 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:16.925127 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda204d-1187-4e34-b8c2-32abae0da076 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:16.925127 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda204d-1187-4e34-b8c2-32abae0da076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:16.958301 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-3fda204d-1187-4e34-b8c2-32abae0da076 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:16.960505 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3fda204d-1187-4e34-b8c2-32abae0da076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:17.058558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1b5f6-146f-49eb-a27e-832b60bd765a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:17.101309 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 57/119] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:17.102954 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda204d-1187-4e34-b8c2-32abae0da076 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:17.113379 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d2adb-c0ca-4939-8052-2db24768ad47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:17.115103 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d2adb-c0ca-4939-8052-2db24768ad47 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:17.115853 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d2adb-c0ca-4939-8052-2db24768ad47 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:17.116613 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d2adb-c0ca-4939-8052-2db24768ad47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:17.136716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d2adb-c0ca-4939-8052-2db24768ad47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:17.151645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-310d2adb-c0ca-4939-8052-2db24768ad47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:24:17.151645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:17.151645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:17.151645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:17.151645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:17.151645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:17.151645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:17.151645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:17.151645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:17.151645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:17.151645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:17.151645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:17.151645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:17.151645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:17.151645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:17.151645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:17.154211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:17.154211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:17.154211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:17.154211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:17.154211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:17.154211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:17.154211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:17.154211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:17.154211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:17.154211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:17.154211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:17.154211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:17.154211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:17.154211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:17.154211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:17.154211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:17.155741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:17.155741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:17.155741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:17.155741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:17.155741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:17.155741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:17.155741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:17.155741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:17.155741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:17.155741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:17.155741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:24:17.155741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:17.159958 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 63/120] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:24:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:17.161876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d2adb-c0ca-4939-8052-2db24768ad47 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:17.207045 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:17.309665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:17.317022 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda204d-1187-4e34-b8c2-32abae0da076 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:17.352528 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d2adb-c0ca-4939-8052-2db24768ad47 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:17.388266 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5540a94-d0b3-4c89-9616-ed1c2de32bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:17.389779 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5540a94-d0b3-4c89-9616-ed1c2de32bde None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:17.390490 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5540a94-d0b3-4c89-9616-ed1c2de32bde None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:17.391177 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5540a94-d0b3-4c89-9616-ed1c2de32bde None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:17.392906 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5540a94-d0b3-4c89-9616-ed1c2de32bde None admin] RBAC: Adding query filter params (name=member) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:17.393744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5540a94-d0b3-4c89-9616-ed1c2de32bde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:17.396585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5540a94-d0b3-4c89-9616-ed1c2de32bde None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:17.414308 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 58/121] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:17.416226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5540a94-d0b3-4c89-9616-ed1c2de32bde None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:17.426276 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e862a-3214-4eee-a3d3-88c7d5fe3569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:17.427468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e862a-3214-4eee-a3d3-88c7d5fe3569 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:17.428014 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e862a-3214-4eee-a3d3-88c7d5fe3569 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:17.428498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e862a-3214-4eee-a3d3-88c7d5fe3569 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:17.430080 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e862a-3214-4eee-a3d3-88c7d5fe3569 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:17.430616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e862a-3214-4eee-a3d3-88c7d5fe3569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:17.434112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e862a-3214-4eee-a3d3-88c7d5fe3569 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:17.455715 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 64/122] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:24:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:17.455715 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e862a-3214-4eee-a3d3-88c7d5fe3569 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:17.627837 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5540a94-d0b3-4c89-9616-ed1c2de32bde None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:17.640177 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e862a-3214-4eee-a3d3-88c7d5fe3569 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:17.649885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:17.703888 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc592e-addc-45d8-bd75-ecfbce97349f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:17.705635 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc592e-addc-45d8-bd75-ecfbce97349f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:17.706537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc592e-addc-45d8-bd75-ecfbce97349f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:17.706692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c286ab-0d82-4eed-a3f3-cb83287e5504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:17.707482 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc592e-addc-45d8-bd75-ecfbce97349f None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:17.708143 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c286ab-0d82-4eed-a3f3-cb83287e5504 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:17.708634 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c286ab-0d82-4eed-a3f3-cb83287e5504 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:17.709122 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c286ab-0d82-4eed-a3f3-cb83287e5504 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:17.710192 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc592e-addc-45d8-bd75-ecfbce97349f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:17.711006 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc592e-addc-45d8-bd75-ecfbce97349f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:17.714953 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc592e-addc-45d8-bd75-ecfbce97349f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:17.722853 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:17.730068 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c286ab-0d82-4eed-a3f3-cb83287e5504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:17.733583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 59/123] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:17.735496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc592e-addc-45d8-bd75-ecfbce97349f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:17.746318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-17c286ab-0d82-4eed-a3f3-cb83287e5504 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:17.746318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:17.746318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:17.746318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:17.746318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:17.746318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:17.746318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:17.746318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:17.746318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:17.746318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:17.746318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:17.746318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:17.746318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:17.746318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:17.746318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:17.746318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:17.748096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:17.748096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:17.748096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:17.748096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:17.748096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:17.748096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:17.748096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:17.748096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:17.748096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:17.748096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:17.748096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:17.748096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:17.748096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:17.748096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:17.748096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:17.748096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:17.749630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:17.749630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:17.749630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:17.749630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:17.749630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:17.749630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:17.749630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:17.749630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:17.749630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:17.749630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:17.749630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:17.749630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:17.749630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 65/124] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:17.757601 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c286ab-0d82-4eed-a3f3-cb83287e5504 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:17.954817 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc592e-addc-45d8-bd75-ecfbce97349f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:17.958765 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c286ab-0d82-4eed-a3f3-cb83287e5504 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:18.021049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf356f6e-6c93-4e9e-a621-b2597e11e919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:18.021347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d03bd1-66e0-4354-8f80-ea20e4aa2ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:18.022455 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf356f6e-6c93-4e9e-a621-b2597e11e919 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:18.022939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf356f6e-6c93-4e9e-a621-b2597e11e919 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:18.023423 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf356f6e-6c93-4e9e-a621-b2597e11e919 None admin] PATH_INFO: `/v3/users/a9e3e8eb9c8c4db0ae0629919a5b1a56` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:18.024008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d03bd1-66e0-4354-8f80-ea20e4aa2ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:18.024898 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d03bd1-66e0-4354-8f80-ea20e4aa2ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:18.025614 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d03bd1-66e0-4354-8f80-ea20e4aa2ab4 None admin] PATH_INFO: `/v3/users/a9e3e8eb9c8c4db0ae0629919a5b1a56` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:18.025762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf356f6e-6c93-4e9e-a621-b2597e11e919 None admin] RBAC: Authorizing `identity:get_user(user_id=a9e3e8eb9c8c4db0ae0629919a5b1a56)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:18.029119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf356f6e-6c93-4e9e-a621-b2597e11e919 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:18.029584 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d03bd1-66e0-4354-8f80-ea20e4aa2ab4 None admin] RBAC: Authorizing `identity:get_user(user_id=a9e3e8eb9c8c4db0ae0629919a5b1a56)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:18.032279 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 66/125] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:17 2022] GET /identity/v3/users/a9e3e8eb9c8c4db0ae0629919a5b1a56 => generated 275 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:18.038078 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf356f6e-6c93-4e9e-a621-b2597e11e919 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:18.039370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d03bd1-66e0-4354-8f80-ea20e4aa2ab4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:18.043553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 60/126] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:17 2022] GET /identity/v3/users/a9e3e8eb9c8c4db0ae0629919a5b1a56 => generated 275 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:18.045371 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d03bd1-66e0-4354-8f80-ea20e4aa2ab4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:18.128009 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:18.170919 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:18.234295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf356f6e-6c93-4e9e-a621-b2597e11e919 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:18.261358 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d03bd1-66e0-4354-8f80-ea20e4aa2ab4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:18.297647 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5e747-2ba6-4fec-9246-4114a1496d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:18.299575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5e747-2ba6-4fec-9246-4114a1496d0d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:18.299806 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5e747-2ba6-4fec-9246-4114a1496d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:18.300300 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5e747-2ba6-4fec-9246-4114a1496d0d None admin] PATH_INFO: `/v3/users/93ad24fe28ec40dbbca364cbdb34972d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:18.302486 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5e747-2ba6-4fec-9246-4114a1496d0d None admin] RBAC: Authorizing `identity:get_user(user_id=93ad24fe28ec40dbbca364cbdb34972d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:18.305217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5e747-2ba6-4fec-9246-4114a1496d0d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:18.308605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 67/127] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:18 2022] GET /identity/v3/users/93ad24fe28ec40dbbca364cbdb34972d => generated 247 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:18.310256 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5e747-2ba6-4fec-9246-4114a1496d0d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:18.334896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fbc20c-e5ca-400c-aaf3-c63b11886bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:18.336573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbc20c-e5ca-400c-aaf3-c63b11886bab None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:18.337466 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbc20c-e5ca-400c-aaf3-c63b11886bab None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:18.338184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbc20c-e5ca-400c-aaf3-c63b11886bab None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:18.341362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fbc20c-e5ca-400c-aaf3-c63b11886bab None admin] RBAC: Adding query filter params (name=member) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:18.342230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fbc20c-e5ca-400c-aaf3-c63b11886bab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:18.345197 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fbc20c-e5ca-400c-aaf3-c63b11886bab None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:18.362328 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 61/128] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:18.369657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fbc20c-e5ca-400c-aaf3-c63b11886bab None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:18.500755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5e747-2ba6-4fec-9246-4114a1496d0d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:18.563194 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a09fb-c3af-40c8-8d12-369dfe484d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:18.564887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a09fb-c3af-40c8-8d12-369dfe484d68 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:18.565300 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a09fb-c3af-40c8-8d12-369dfe484d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:18.565802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a09fb-c3af-40c8-8d12-369dfe484d68 None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:18.568428 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a09fb-c3af-40c8-8d12-369dfe484d68 None admin] RBAC: Authorizing `identity:get_project(project_id=791daf53d0364b258d9cfb153ffa3111)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:18.570276 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:18.571385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a09fb-c3af-40c8-8d12-369dfe484d68 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:18.574127 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 68/129] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:18 2022] GET /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111 => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:18.575528 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a09fb-c3af-40c8-8d12-369dfe484d68 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:18.577927 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fbc20c-e5ca-400c-aaf3-c63b11886bab None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:18.588298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:18.636661 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faea352-b90a-42ef-b4d3-ef1a4a45ba8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:18.637963 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faea352-b90a-42ef-b4d3-ef1a4a45ba8e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:18.638519 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faea352-b90a-42ef-b4d3-ef1a4a45ba8e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:18.639105 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faea352-b90a-42ef-b4d3-ef1a4a45ba8e None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:18.640770 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faea352-b90a-42ef-b4d3-ef1a4a45ba8e None admin] RBAC: Authorizing `identity:get_project(project_id=791daf53d0364b258d9cfb153ffa3111)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:18.642584 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faea352-b90a-42ef-b4d3-ef1a4a45ba8e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:18.644964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 62/130] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:18 2022] GET /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111 => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:18.650728 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faea352-b90a-42ef-b4d3-ef1a4a45ba8e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:18.741021 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a09fb-c3af-40c8-8d12-369dfe484d68 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:18.788087 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faea352-b90a-42ef-b4d3-ef1a4a45ba8e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:18.790194 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac87319-024e-4889-9520-05784fd088f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:18.791065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac87319-024e-4889-9520-05784fd088f6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:18.792985 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac87319-024e-4889-9520-05784fd088f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:18.792985 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac87319-024e-4889-9520-05784fd088f6 None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:18.794394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac87319-024e-4889-9520-05784fd088f6 None admin] RBAC: Authorizing `identity:get_project(project_id=791daf53d0364b258d9cfb153ffa3111)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:18.798563 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac87319-024e-4889-9520-05784fd088f6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:18.800713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 69/131] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:18 2022] GET /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:18.801252 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac87319-024e-4889-9520-05784fd088f6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:18.855774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5addc9-6b71-4a46-8186-9b6e4e5d7359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:18.857056 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5addc9-6b71-4a46-8186-9b6e4e5d7359 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:18.858697 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5addc9-6b71-4a46-8186-9b6e4e5d7359 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:18.859174 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5addc9-6b71-4a46-8186-9b6e4e5d7359 None admin] PATH_INFO: `/v3/users/a9e3e8eb9c8c4db0ae0629919a5b1a56` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:18.861882 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5addc9-6b71-4a46-8186-9b6e4e5d7359 None admin] RBAC: Authorizing `identity:get_user(user_id=a9e3e8eb9c8c4db0ae0629919a5b1a56)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:18.869157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5addc9-6b71-4a46-8186-9b6e4e5d7359 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:18.873651 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 63/132] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:18 2022] GET /identity/v3/users/a9e3e8eb9c8c4db0ae0629919a5b1a56 => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:18.874804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5addc9-6b71-4a46-8186-9b6e4e5d7359 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:18.947749 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:18.957678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:18.997945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac87319-024e-4889-9520-05784fd088f6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:19.047517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be4e8a-e7b8-4382-af15-c0e982c24cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:19.049111 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be4e8a-e7b8-4382-af15-c0e982c24cf3 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:19.049633 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be4e8a-e7b8-4382-af15-c0e982c24cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:19.050103 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be4e8a-e7b8-4382-af15-c0e982c24cf3 None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111/users/a9e3e8eb9c8c4db0ae0629919a5b1a56/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:19.053354 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be4e8a-e7b8-4382-af15-c0e982c24cf3 None admin] RBAC: Authorizing `identity:create_grant(project_id=791daf53d0364b258d9cfb153ffa3111, user_id=a9e3e8eb9c8c4db0ae0629919a5b1a56, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:19.055967 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be4e8a-e7b8-4382-af15-c0e982c24cf3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:19.071916 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5addc9-6b71-4a46-8186-9b6e4e5d7359 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:19.114656 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b0ee4-1c4b-4b43-8f91-6dced2319728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:19.115581 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b0ee4-1c4b-4b43-8f91-6dced2319728 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:19.115898 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b0ee4-1c4b-4b43-8f91-6dced2319728 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:19.116207 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b0ee4-1c4b-4b43-8f91-6dced2319728 None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111/users/a9e3e8eb9c8c4db0ae0629919a5b1a56/roles/d2ba35e60f974010a5e7dd5c4bebecf8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:19.120097 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b0ee4-1c4b-4b43-8f91-6dced2319728 None admin] RBAC: Authorizing `identity:create_grant(project_id=791daf53d0364b258d9cfb153ffa3111, user_id=a9e3e8eb9c8c4db0ae0629919a5b1a56, role_id=d2ba35e60f974010a5e7dd5c4bebecf8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:19.122819 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b0ee4-1c4b-4b43-8f91-6dced2319728 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:19.134587 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 70/133] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:24:18 2022] PUT /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111/users/a9e3e8eb9c8c4db0ae0629919a5b1a56/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:24:19.138603 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c6c16-0aaf-4293-af21-b5caf069172d None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:19.139130 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c6c16-0aaf-4293-af21-b5caf069172d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:19.139635 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c6c16-0aaf-4293-af21-b5caf069172d None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:19.143049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 71/134] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:19.147787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c6c16-0aaf-4293-af21-b5caf069172d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:19.219214 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 64/135] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:24:18 2022] PUT /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111/users/a9e3e8eb9c8c4db0ae0629919a5b1a56/roles/d2ba35e60f974010a5e7dd5c4bebecf8 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:24:19.220336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b0ee4-1c4b-4b43-8f91-6dced2319728 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:19.448916 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c6c16-0aaf-4293-af21-b5caf069172d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:19.487103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:19.500594 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b0ee4-1c4b-4b43-8f91-6dced2319728 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:19.511230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:19.521730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5eead6-e8a3-4107-a58b-830798084e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:19.523057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5eead6-e8a3-4107-a58b-830798084e24 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:19.523570 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5eead6-e8a3-4107-a58b-830798084e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:19.530448 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5eead6-e8a3-4107-a58b-830798084e24 None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111/users/93ad24fe28ec40dbbca364cbdb34972d/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:19.530448 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5eead6-e8a3-4107-a58b-830798084e24 None admin] RBAC: Authorizing `identity:create_grant(project_id=791daf53d0364b258d9cfb153ffa3111, user_id=93ad24fe28ec40dbbca364cbdb34972d, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:19.531197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5eead6-e8a3-4107-a58b-830798084e24 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:19.569834 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83830222-a071-4094-b8d3-52d2f2453e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:19.571289 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83830222-a071-4094-b8d3-52d2f2453e69 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:19.571814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83830222-a071-4094-b8d3-52d2f2453e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:19.572303 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83830222-a071-4094-b8d3-52d2f2453e69 None admin] PATH_INFO: `/v3/projects/4c4224e757934016a179499c9c15cf75` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:19.574409 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83830222-a071-4094-b8d3-52d2f2453e69 None admin] RBAC: Authorizing `identity:get_project(project_id=4c4224e757934016a179499c9c15cf75)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:19.577036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83830222-a071-4094-b8d3-52d2f2453e69 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:19.580499 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 65/136] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:19 2022] GET /identity/v3/projects/4c4224e757934016a179499c9c15cf75 => generated 312 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:19.582526 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6b1dc-1883-4540-953e-963ac233f9ee None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:19.585507 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6b1dc-1883-4540-953e-963ac233f9ee None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:19.585507 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6b1dc-1883-4540-953e-963ac233f9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:19.614082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-7de6b1dc-1883-4540-953e-963ac233f9ee None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:19.616178 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7de6b1dc-1883-4540-953e-963ac233f9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:19.650281 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 72/137] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:24:19 2022] PUT /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111/users/93ad24fe28ec40dbbca364cbdb34972d/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:24:19.653586 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922b241-445b-4711-aeab-e23795eaba98 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:19.654048 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922b241-445b-4711-aeab-e23795eaba98 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:19.654489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922b241-445b-4711-aeab-e23795eaba98 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:19.658417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 73/138] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:19.660040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922b241-445b-4711-aeab-e23795eaba98 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:19.796568 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 66/139] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:19.798336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1588e-9a23-41d0-9b7d-98fe8497881e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:19.798856 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1588e-9a23-41d0-9b7d-98fe8497881e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:19.800044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1588e-9a23-41d0-9b7d-98fe8497881e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:19.862831 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-f7e1588e-9a23-41d0-9b7d-98fe8497881e None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:19.864904 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f7e1588e-9a23-41d0-9b7d-98fe8497881e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:19.987010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922b241-445b-4711-aeab-e23795eaba98 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:20.027565 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 67/140] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:20.029616 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1588e-9a23-41d0-9b7d-98fe8497881e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:20.078593 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:20.079543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a458964-ef25-4af6-b1fa-16f9bbd6f9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:20.082005 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a458964-ef25-4af6-b1fa-16f9bbd6f9e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:20.082750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a458964-ef25-4af6-b1fa-16f9bbd6f9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:20.083434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a458964-ef25-4af6-b1fa-16f9bbd6f9e5 None admin] PATH_INFO: `/v3/projects/4c4224e757934016a179499c9c15cf75/users/a9e3e8eb9c8c4db0ae0629919a5b1a56/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:20.087634 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a458964-ef25-4af6-b1fa-16f9bbd6f9e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c4224e757934016a179499c9c15cf75, user_id=a9e3e8eb9c8c4db0ae0629919a5b1a56, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:20.091229 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a458964-ef25-4af6-b1fa-16f9bbd6f9e5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:20.207213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 74/141] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:24:19 2022] PUT /identity/v3/projects/4c4224e757934016a179499c9c15cf75/users/a9e3e8eb9c8c4db0ae0629919a5b1a56/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:24:20.209518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a458964-ef25-4af6-b1fa-16f9bbd6f9e5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:20.285246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1588e-9a23-41d0-9b7d-98fe8497881e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:20.367516 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e574c3-23ca-46eb-a0e5-a1df220bd696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:20.368829 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e574c3-23ca-46eb-a0e5-a1df220bd696 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:20.369334 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e574c3-23ca-46eb-a0e5-a1df220bd696 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:20.369879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e574c3-23ca-46eb-a0e5-a1df220bd696 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:20.377258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:20.381222 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e574c3-23ca-46eb-a0e5-a1df220bd696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:20.384118 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e574c3-23ca-46eb-a0e5-a1df220bd696 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:20.387955 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 68/142] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:24:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:20.396989 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e574c3-23ca-46eb-a0e5-a1df220bd696 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:20.557065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a458964-ef25-4af6-b1fa-16f9bbd6f9e5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:20.654058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df64f6-0959-4fcf-bc7d-12b7ddc3ccf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:20.655324 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df64f6-0959-4fcf-bc7d-12b7ddc3ccf9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:20.655864 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df64f6-0959-4fcf-bc7d-12b7ddc3ccf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:20.656362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df64f6-0959-4fcf-bc7d-12b7ddc3ccf9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:20.661842 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:20.671624 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e574c3-23ca-46eb-a0e5-a1df220bd696 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:20.682447 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df64f6-0959-4fcf-bc7d-12b7ddc3ccf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:20.706596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-54df64f6-0959-4fcf-bc7d-12b7ddc3ccf9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 08:24:20.706596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:20.706596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:20.706596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:20.706596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:20.706596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:20.706596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:20.706596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:20.706596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:20.706596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:20.706596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:20.706596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:20.706596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:20.706596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:20.706596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:20.706596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:20.708240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:20.708240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:20.708240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:20.708240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:20.708240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:20.708240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:20.708240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:20.708240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:20.708240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:20.708240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:20.708240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:20.708240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:20.708240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:20.708240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:20.708240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:20.708240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:20.709664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:20.709664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:20.709664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:20.709664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:20.709664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:20.709664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:20.709664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:20.709664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:20.709664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:20.709664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:20.709664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 08:24:20.709664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:20.709664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 75/143] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 501 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:20.717932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2bfae-7f2f-407b-b23c-f97245c12c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:20.718642 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2bfae-7f2f-407b-b23c-f97245c12c4b None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:20.718936 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2bfae-7f2f-407b-b23c-f97245c12c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:20.719220 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2bfae-7f2f-407b-b23c-f97245c12c4b None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:20.720006 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2bfae-7f2f-407b-b23c-f97245c12c4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:20.721525 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2bfae-7f2f-407b-b23c-f97245c12c4b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:20.722470 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df64f6-0959-4fcf-bc7d-12b7ddc3ccf9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:20.832328 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 69/144] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 18 08:24:20 2022] POST /identity/v3/users => generated 283 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:24:20.954916 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df64f6-0959-4fcf-bc7d-12b7ddc3ccf9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:20.976379 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:21.022417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af54a2-9201-4fc7-a8d2-4ca64115ee5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:21.023490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af54a2-9201-4fc7-a8d2-4ca64115ee5a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:21.023995 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af54a2-9201-4fc7-a8d2-4ca64115ee5a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:21.024449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af54a2-9201-4fc7-a8d2-4ca64115ee5a None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:21.025682 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af54a2-9201-4fc7-a8d2-4ca64115ee5a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:21.026297 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af54a2-9201-4fc7-a8d2-4ca64115ee5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:21.029521 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af54a2-9201-4fc7-a8d2-4ca64115ee5a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:21.051844 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 76/145] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:24:21.055535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2bfae-7f2f-407b-b23c-f97245c12c4b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:21.288474 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2bfae-7f2f-407b-b23c-f97245c12c4b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:21.348219 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdacc6-d995-47e8-bd82-79562b92e1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:21.349559 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdacc6-d995-47e8-bd82-79562b92e1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:21.350025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdacc6-d995-47e8-bd82-79562b92e1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:21.350476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdacc6-d995-47e8-bd82-79562b92e1a5 None admin] PATH_INFO: `/v3/users/ab00423871a34113bf9e5853f83e3011` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:21.352414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdacc6-d995-47e8-bd82-79562b92e1a5 None admin] RBAC: Authorizing `identity:get_user(user_id=ab00423871a34113bf9e5853f83e3011)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:21.355340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdacc6-d995-47e8-bd82-79562b92e1a5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:21.357882 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 70/146] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:21 2022] GET /identity/v3/users/ab00423871a34113bf9e5853f83e3011 => generated 289 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:21.364379 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af54a2-9201-4fc7-a8d2-4ca64115ee5a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:21.564896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af54a2-9201-4fc7-a8d2-4ca64115ee5a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:21.631622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed656b2e-e14d-432a-957c-90cb1cf73200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:21.633095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed656b2e-e14d-432a-957c-90cb1cf73200 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:21.633596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed656b2e-e14d-432a-957c-90cb1cf73200 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:21.634084 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed656b2e-e14d-432a-957c-90cb1cf73200 None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:21.637016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed656b2e-e14d-432a-957c-90cb1cf73200 None admin] RBAC: Authorizing `identity:get_project(project_id=791daf53d0364b258d9cfb153ffa3111)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:21.639878 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed656b2e-e14d-432a-957c-90cb1cf73200 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:21.643354 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 77/147] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:21 2022] GET /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111 => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:24:21.653032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdacc6-d995-47e8-bd82-79562b92e1a5 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:21.688884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:21.903437 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdacc6-d995-47e8-bd82-79562b92e1a5 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:21.976023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812f232-85bb-4d9f-b576-b40f9b05accb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:21.977617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812f232-85bb-4d9f-b576-b40f9b05accb None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:21.978111 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812f232-85bb-4d9f-b576-b40f9b05accb None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:21.978642 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812f232-85bb-4d9f-b576-b40f9b05accb None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111/users/ab00423871a34113bf9e5853f83e3011/roles/24179486bd6a4eb985a83d47ca5c2b4f` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:21.986934 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d812f232-85bb-4d9f-b576-b40f9b05accb None admin] RBAC: Authorizing `identity:create_grant(project_id=791daf53d0364b258d9cfb153ffa3111, user_id=ab00423871a34113bf9e5853f83e3011, role_id=24179486bd6a4eb985a83d47ca5c2b4f)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:21.989603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d812f232-85bb-4d9f-b576-b40f9b05accb None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:22.088552 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 71/148] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:24:21 2022] PUT /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111/users/ab00423871a34113bf9e5853f83e3011/roles/24179486bd6a4eb985a83d47ca5c2b4f => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:24:22.883287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c0ed8-f853-4ae0-bdcc-da825b546dac None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:22.883820 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c0ed8-f853-4ae0-bdcc-da825b546dac None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:22.884177 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c0ed8-f853-4ae0-bdcc-da825b546dac None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:22.887351 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 78/149] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:22.895962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e7b87-c5c9-4017-a565-3f3d10a35141 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:22.896730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e7b87-c5c9-4017-a565-3f3d10a35141 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:22.897033 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e7b87-c5c9-4017-a565-3f3d10a35141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:22.932195 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-410e7b87-c5c9-4017-a565-3f3d10a35141 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:22.934285 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-410e7b87-c5c9-4017-a565-3f3d10a35141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:23.088024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 72/150] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:24:23.147858 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c0ed8-f853-4ae0-bdcc-da825b546dac None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:23.233829 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:23.377005 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c0ed8-f853-4ae0-bdcc-da825b546dac None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:23.463148 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8bb5c9-4d84-4782-99a4-c4cbc5d0ee39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:23.465769 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bb5c9-4d84-4782-99a4-c4cbc5d0ee39 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:23.466261 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bb5c9-4d84-4782-99a4-c4cbc5d0ee39 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:23.466743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bb5c9-4d84-4782-99a4-c4cbc5d0ee39 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:23.496347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8bb5c9-4d84-4782-99a4-c4cbc5d0ee39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:23.515994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-4d8bb5c9-4d84-4782-99a4-c4cbc5d0ee39 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:23.515994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:23.515994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:23.515994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:23.515994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:23.515994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:23.515994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:23.515994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:23.515994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:23.515994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:23.515994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:23.515994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:23.515994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:23.515994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:23.515994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:23.515994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:23.517645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:23.517645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:23.517645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:23.517645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:23.517645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:23.517645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:23.517645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:23.517645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:23.517645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:23.517645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:23.517645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:23.517645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:23.517645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:23.517645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:23.517645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:23.517645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:23.519158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:23.519158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:23.519158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:23.519158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:23.519158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:23.519158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:23.519158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:23.519158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:23.519158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:23.519158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:23.519158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:23.519158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:23.519158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 79/151] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:24:23.527049 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e7b87-c5c9-4017-a565-3f3d10a35141 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:23.558557 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:23.728978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e7b87-c5c9-4017-a565-3f3d10a35141 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:23.757826 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a2e33-0eda-4cb5-ba82-1ac5487c1f9e None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:23.758267 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a2e33-0eda-4cb5-ba82-1ac5487c1f9e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:23.758751 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a2e33-0eda-4cb5-ba82-1ac5487c1f9e None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:23.762268 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 80/152] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:23.772773 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7698e-140a-4be3-b2d6-b5c7b3e7833c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:23.773234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7698e-140a-4be3-b2d6-b5c7b3e7833c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:23.773704 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7698e-140a-4be3-b2d6-b5c7b3e7833c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:23.837529 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-9fe7698e-140a-4be3-b2d6-b5c7b3e7833c None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:23.843698 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9fe7698e-140a-4be3-b2d6-b5c7b3e7833c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:23.860902 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f1a19-8740-4b30-92c4-b3cbb01cfbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:23.862013 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f1a19-8740-4b30-92c4-b3cbb01cfbd5 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:23.862568 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f1a19-8740-4b30-92c4-b3cbb01cfbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:23.863057 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f1a19-8740-4b30-92c4-b3cbb01cfbd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:23.864491 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f1a19-8740-4b30-92c4-b3cbb01cfbd5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:23.865029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f1a19-8740-4b30-92c4-b3cbb01cfbd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:23.867930 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f1a19-8740-4b30-92c4-b3cbb01cfbd5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:23.892783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 73/153] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:24:23.899393 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f1a19-8740-4b30-92c4-b3cbb01cfbd5 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:24.051816 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 81/154] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:24.065375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:24.136135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7698e-140a-4be3-b2d6-b5c7b3e7833c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:24.145056 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f1a19-8740-4b30-92c4-b3cbb01cfbd5 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:24.244288 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd785d-12ac-4398-b200-39693e60d9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:24.245629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd785d-12ac-4398-b200-39693e60d9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:24.246134 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd785d-12ac-4398-b200-39693e60d9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:24.246713 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd785d-12ac-4398-b200-39693e60d9d9 None admin] PATH_INFO: `/v3/users/a9e3e8eb9c8c4db0ae0629919a5b1a56` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:24.248187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:24.255949 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd785d-12ac-4398-b200-39693e60d9d9 None admin] RBAC: Authorizing `identity:get_user(user_id=a9e3e8eb9c8c4db0ae0629919a5b1a56)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:24.257851 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd785d-12ac-4398-b200-39693e60d9d9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:24.260573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 74/155] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:23 2022] GET /identity/v3/users/a9e3e8eb9c8c4db0ae0629919a5b1a56 => generated 275 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:24:24.261369 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de9058-e048-4f9f-9a16-633c88ca8172 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:24.261763 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de9058-e048-4f9f-9a16-633c88ca8172 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:24.262145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de9058-e048-4f9f-9a16-633c88ca8172 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:24.263911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 75/156] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:24.273121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de9058-e048-4f9f-9a16-633c88ca8172 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:24.444798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7698e-140a-4be3-b2d6-b5c7b3e7833c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:24.561592 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de9058-e048-4f9f-9a16-633c88ca8172 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:24.562425 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f6100-af76-40a4-818c-030350b7a341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:24.563781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f6100-af76-40a4-818c-030350b7a341 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:24.564190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f6100-af76-40a4-818c-030350b7a341 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:24.564583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f6100-af76-40a4-818c-030350b7a341 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:24.591718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f6100-af76-40a4-818c-030350b7a341 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:24.624950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-649f6100-af76-40a4-818c-030350b7a341 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:24:24.624950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:24.624950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:24.624950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:24.624950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:24.624950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:24.624950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:24.624950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:24.624950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:24.624950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:24.624950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:24.624950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:24.624950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:24.624950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:24.624950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:24.624950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:24.626636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:24.626636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:24.626636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:24.626636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:24.626636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:24.626636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:24.626636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:24.626636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:24.626636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:24.626636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:24.626636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:24.626636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:24.626636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:24.626636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:24.626636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:24.626636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:24.628028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:24.628028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:24.628028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:24.628028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:24.628028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:24.628028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:24.628028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:24.628028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:24.628028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:24.628028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:24.628028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:24:24.628028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:24.628028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 82/157] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:24:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 492 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:24:24.630372 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad420f6-1e35-4725-92ab-4390539c7a84 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:24.630372 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad420f6-1e35-4725-92ab-4390539c7a84 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:24.630372 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad420f6-1e35-4725-92ab-4390539c7a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:24.650376 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823bbfe-2c51-4ad8-86e1-41f816173779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:24.651184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823bbfe-2c51-4ad8-86e1-41f816173779 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:24.651467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823bbfe-2c51-4ad8-86e1-41f816173779 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:24.651761 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823bbfe-2c51-4ad8-86e1-41f816173779 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:24.659121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:24.681658 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-aad420f6-1e35-4725-92ab-4390539c7a84 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:24.684366 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-aad420f6-1e35-4725-92ab-4390539c7a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:24.826217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3823bbfe-2c51-4ad8-86e1-41f816173779 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:24.827653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3823bbfe-2c51-4ad8-86e1-41f816173779 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:24.910456 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 83/158] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:24.912116 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad420f6-1e35-4725-92ab-4390539c7a84 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:25.000224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:25.179845 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 76/159] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 907 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 08:24:25.180781 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823bbfe-2c51-4ad8-86e1-41f816173779 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:25.251454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad420f6-1e35-4725-92ab-4390539c7a84 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:25.284291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:25.349342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053d63e-1d98-459d-b0c0-16c1a407bdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:25.350067 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053d63e-1d98-459d-b0c0-16c1a407bdfb None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:25.350353 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053d63e-1d98-459d-b0c0-16c1a407bdfb None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:25.350632 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053d63e-1d98-459d-b0c0-16c1a407bdfb None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:25.351427 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053d63e-1d98-459d-b0c0-16c1a407bdfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:25.351770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053d63e-1d98-459d-b0c0-16c1a407bdfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:25.353258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053d63e-1d98-459d-b0c0-16c1a407bdfb None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:25.389125 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 84/160] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 478 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:25.390018 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053d63e-1d98-459d-b0c0-16c1a407bdfb None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:25.507707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823bbfe-2c51-4ad8-86e1-41f816173779 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:25.667357 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e11e9-40df-4ee7-885a-4fb61fa94269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:25.675449 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e11e9-40df-4ee7-885a-4fb61fa94269 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:25.684127 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e11e9-40df-4ee7-885a-4fb61fa94269 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:25.684554 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e11e9-40df-4ee7-885a-4fb61fa94269 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:25.724141 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e11e9-40df-4ee7-885a-4fb61fa94269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:25.744273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-3b1e11e9-40df-4ee7-885a-4fb61fa94269 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:24:25.744273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:25.744273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:25.744273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:25.744273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:25.744273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:25.744273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:25.744273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:25.744273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:25.744273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:25.744273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:25.744273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:25.744273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:25.744273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:25.744273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:25.744273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:25.748231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:25.748231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:25.748231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:25.748231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:25.748231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:25.748231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:25.748231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:25.748231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:25.748231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:25.748231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:25.748231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:25.748231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:25.748231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:25.748231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:25.748231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:25.748231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:25.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:25.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:25.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:25.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:25.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:25.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:25.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:25.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:25.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:25.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:25.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:24:25.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:24:25.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 77/161] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:24:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 566 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:25.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e11e9-40df-4ee7-885a-4fb61fa94269 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:25.790949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:25.901424 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053d63e-1d98-459d-b0c0-16c1a407bdfb None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:26.029760 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e4bd5-f823-4c61-8ac6-9919888cc9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:26.031817 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e4bd5-f823-4c61-8ac6-9919888cc9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:26.032551 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e4bd5-f823-4c61-8ac6-9919888cc9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:26.033265 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e4bd5-f823-4c61-8ac6-9919888cc9c1 None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:26.043932 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e4bd5-f823-4c61-8ac6-9919888cc9c1 None admin] RBAC: Authorizing `identity:get_project(project_id=791daf53d0364b258d9cfb153ffa3111)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:26.047111 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e4bd5-f823-4c61-8ac6-9919888cc9c1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:26.050564 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 85/162] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:25 2022] GET /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111 => generated 298 bytes in 661 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:26.056567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e4bd5-f823-4c61-8ac6-9919888cc9c1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:26.152086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:26.203430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e11e9-40df-4ee7-885a-4fb61fa94269 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:26.355349 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca885689-55ac-4e75-ac7c-650c02596a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:26.356812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca885689-55ac-4e75-ac7c-650c02596a2d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:26.357307 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca885689-55ac-4e75-ac7c-650c02596a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:26.358161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca885689-55ac-4e75-ac7c-650c02596a2d None admin] PATH_INFO: `/v3/users/93ad24fe28ec40dbbca364cbdb34972d` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:26.359760 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca885689-55ac-4e75-ac7c-650c02596a2d None admin] RBAC: Authorizing `identity:get_user(user_id=93ad24fe28ec40dbbca364cbdb34972d)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:26.361453 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca885689-55ac-4e75-ac7c-650c02596a2d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:26.369580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 78/163] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:25 2022] GET /identity/v3/users/93ad24fe28ec40dbbca364cbdb34972d => generated 247 bytes in 621 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:26.373016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790a0e7-808b-46a3-bc2e-f6a362a3c07d None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:26.373910 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790a0e7-808b-46a3-bc2e-f6a362a3c07d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:26.374738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790a0e7-808b-46a3-bc2e-f6a362a3c07d None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:26.377829 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 79/164] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:26.379048 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790a0e7-808b-46a3-bc2e-f6a362a3c07d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:26.442385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e4bd5-f823-4c61-8ac6-9919888cc9c1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:26.540903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4488a5-f3dc-4dc2-92e7-2d5ebd93785c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:26.542166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4488a5-f3dc-4dc2-92e7-2d5ebd93785c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:26.542842 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4488a5-f3dc-4dc2-92e7-2d5ebd93785c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:26.543355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4488a5-f3dc-4dc2-92e7-2d5ebd93785c None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:26.548692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4488a5-f3dc-4dc2-92e7-2d5ebd93785c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:26.549325 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4488a5-f3dc-4dc2-92e7-2d5ebd93785c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:26.551352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4488a5-f3dc-4dc2-92e7-2d5ebd93785c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:26.567945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 86/165] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:24:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 516 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:26.569208 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4488a5-f3dc-4dc2-92e7-2d5ebd93785c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:26.599179 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:26.680906 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790a0e7-808b-46a3-bc2e-f6a362a3c07d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:26.776603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee3f1f-9260-4b27-9615-be63debd7e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:26.777920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee3f1f-9260-4b27-9615-be63debd7e8f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:26.778398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee3f1f-9260-4b27-9615-be63debd7e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:26.778883 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee3f1f-9260-4b27-9615-be63debd7e8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:26.780701 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee3f1f-9260-4b27-9615-be63debd7e8f None admin] RBAC: Adding query filter params (role.id=587af95739954600bbf2411974601f74, scope.project.id=791daf53d0364b258d9cfb153ffa3111, user.id=a9e3e8eb9c8c4db0ae0629919a5b1a56) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:26.781260 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee3f1f-9260-4b27-9615-be63debd7e8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:26.784112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee3f1f-9260-4b27-9615-be63debd7e8f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:26.798339 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 80/166] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:24:26 2022] GET /identity/v3/role_assignments?user.id=a9e3e8eb9c8c4db0ae0629919a5b1a56&scope.project.id=791daf53d0364b258d9cfb153ffa3111&role.id=587af95739954600bbf2411974601f74 => generated 608 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:26.800807 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dc94b-300f-47cf-a9d1-a285f6344cb5 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:26.801310 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dc94b-300f-47cf-a9d1-a285f6344cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:26.801789 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dc94b-300f-47cf-a9d1-a285f6344cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:26.835959 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4488a5-f3dc-4dc2-92e7-2d5ebd93785c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:26.843702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-751dc94b-300f-47cf-a9d1-a285f6344cb5 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:26.855768 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-751dc94b-300f-47cf-a9d1-a285f6344cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:26.898893 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb8194-093f-4aa5-bf84-a0d5c817cd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:26.900445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb8194-093f-4aa5-bf84-a0d5c817cd7a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:26.901648 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb8194-093f-4aa5-bf84-a0d5c817cd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:26.902189 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb8194-093f-4aa5-bf84-a0d5c817cd7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:26.955475 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:27.077103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 81/167] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:27.077992 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751dc94b-300f-47cf-a9d1-a285f6344cb5 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:27.090951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb8194-093f-4aa5-bf84-a0d5c817cd7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:27.092522 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb8194-093f-4aa5-bf84-a0d5c817cd7a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:27.116320 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:27.265064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 87/168] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:24:27.266185 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0036521-57aa-4089-a946-e6eac9d10de9 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:27.266468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0036521-57aa-4089-a946-e6eac9d10de9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:27.266758 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0036521-57aa-4089-a946-e6eac9d10de9 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:27.268153 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 88/169] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:27.269322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb5aa7-72c6-439b-9eba-b7dcd25be269 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:27.269604 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb5aa7-72c6-439b-9eba-b7dcd25be269 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:27.271973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb5aa7-72c6-439b-9eba-b7dcd25be269 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:27.273905 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 89/170] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:27.275069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb5aa7-72c6-439b-9eba-b7dcd25be269 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:27.292411 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751dc94b-300f-47cf-a9d1-a285f6344cb5 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:27.335764 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d859ca-812f-49de-b7e5-5fdd366fcb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:27.336655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d859ca-812f-49de-b7e5-5fdd366fcb31 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:27.336933 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d859ca-812f-49de-b7e5-5fdd366fcb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:27.337203 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d859ca-812f-49de-b7e5-5fdd366fcb31 None admin] PATH_INFO: `/v3/users/a9e3e8eb9c8c4db0ae0629919a5b1a56` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:27.339121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d859ca-812f-49de-b7e5-5fdd366fcb31 None admin] RBAC: Authorizing `identity:get_user(user_id=a9e3e8eb9c8c4db0ae0629919a5b1a56)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:27.340953 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d859ca-812f-49de-b7e5-5fdd366fcb31 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:27.342819 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 82/171] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:27 2022] GET /identity/v3/users/a9e3e8eb9c8c4db0ae0629919a5b1a56 => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:27.348055 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92426755-19a7-4a94-9aa0-0b91ccb63310 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:27.348055 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92426755-19a7-4a94-9aa0-0b91ccb63310 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:27.348055 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92426755-19a7-4a94-9aa0-0b91ccb63310 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:27.348055 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 83/172] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:27.348055 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92426755-19a7-4a94-9aa0-0b91ccb63310 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:27.395786 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:27.476183 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb5aa7-72c6-439b-9eba-b7dcd25be269 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:27.491343 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92426755-19a7-4a94-9aa0-0b91ccb63310 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:27.502639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:27.537337 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee8a3e-0eb3-4e9e-9c18-c5352dbe422e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:27.538270 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee8a3e-0eb3-4e9e-9c18-c5352dbe422e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:27.538790 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee8a3e-0eb3-4e9e-9c18-c5352dbe422e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:27.539235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee8a3e-0eb3-4e9e-9c18-c5352dbe422e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:27.559545 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45804f1e-f1f8-403b-98e0-2cb1a8d611b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:27.560205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee8a3e-0eb3-4e9e-9c18-c5352dbe422e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:27.560948 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45804f1e-f1f8-403b-98e0-2cb1a8d611b6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:27.561402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45804f1e-f1f8-403b-98e0-2cb1a8d611b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:27.561842 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45804f1e-f1f8-403b-98e0-2cb1a8d611b6 None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:27.563891 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45804f1e-f1f8-403b-98e0-2cb1a8d611b6 None admin] RBAC: Authorizing `identity:get_project(project_id=791daf53d0364b258d9cfb153ffa3111)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:27.566375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45804f1e-f1f8-403b-98e0-2cb1a8d611b6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:27.569382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 84/173] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:27 2022] GET /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:27.576010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c91f33-53b7-4edc-b521-448580bc6c71 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:27.576010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c91f33-53b7-4edc-b521-448580bc6c71 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:27.576010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c91f33-53b7-4edc-b521-448580bc6c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:27.576334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-42ee8a3e-0eb3-4e9e-9c18-c5352dbe422e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:27.576334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:27.576334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:27.576334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:27.576334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:27.576334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:27.576334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:27.576334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:27.576334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:27.576334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:27.576334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:27.576334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:27.576334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:27.576334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:27.576334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:27.576334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:27.577150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:27.577150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:27.577150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:27.577150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:27.577150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:27.577150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:27.577150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:27.577150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:27.577150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:27.577150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:27.577150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:27.577150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:27.577150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:27.577150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:27.577150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:27.577150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:27.578110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:27.578110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:27.578110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:27.578110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:27.578110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:27.578110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:27.578110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:27.578110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:27.578110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:27.578110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:27.578110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:27.578110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:27.578110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 90/174] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:27.578110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb873a04-83ae-441c-9edb-0f19a58829c5 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:27.578838 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb873a04-83ae-441c-9edb-0f19a58829c5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:27.578838 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb873a04-83ae-441c-9edb-0f19a58829c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:27.606118 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-68c91f33-53b7-4edc-b521-448580bc6c71 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:27.607390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-bb873a04-83ae-441c-9edb-0f19a58829c5 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:27.607723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-68c91f33-53b7-4edc-b521-448580bc6c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:27.609114 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bb873a04-83ae-441c-9edb-0f19a58829c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:27.696123 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 85/175] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:27.697585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c91f33-53b7-4edc-b521-448580bc6c71 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:27.746500 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:27.758927 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 91/176] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:27.761567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8db893-7030-431b-96f4-a2cb3216ba2e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:27.762064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8db893-7030-431b-96f4-a2cb3216ba2e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:27.762555 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8db893-7030-431b-96f4-a2cb3216ba2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:27.793087 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1d8db893-7030-431b-96f4-a2cb3216ba2e None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:27.795006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1d8db893-7030-431b-96f4-a2cb3216ba2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:27.866877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c91f33-53b7-4edc-b521-448580bc6c71 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:27.907922 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6e511-3685-4c37-a8c3-88aa4cce4b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:27.908587 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6e511-3685-4c37-a8c3-88aa4cce4b11 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:27.908868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6e511-3685-4c37-a8c3-88aa4cce4b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:27.909146 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6e511-3685-4c37-a8c3-88aa4cce4b11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:27.922912 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 92/177] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:27.924441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8db893-7030-431b-96f4-a2cb3216ba2e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:27.976768 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:28.000017 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6e511-3685-4c37-a8c3-88aa4cce4b11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:28.001512 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6e511-3685-4c37-a8c3-88aa4cce4b11 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:28.028723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:28.098128 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8db893-7030-431b-96f4-a2cb3216ba2e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:28.118076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 86/178] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:24:28.119354 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6e511-3685-4c37-a8c3-88aa4cce4b11 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:28.146289 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543aa5c-717f-40b7-a19d-61c22f155f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:28.147357 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543aa5c-717f-40b7-a19d-61c22f155f77 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:28.147850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543aa5c-717f-40b7-a19d-61c22f155f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:28.148293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543aa5c-717f-40b7-a19d-61c22f155f77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:28.149556 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543aa5c-717f-40b7-a19d-61c22f155f77 None admin] RBAC: Adding query filter params (role.id=467d49b8a0d3435aaba58464cc8ef85c, scope.project.id=791daf53d0364b258d9cfb153ffa3111, user.id=93ad24fe28ec40dbbca364cbdb34972d) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:28.150062 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543aa5c-717f-40b7-a19d-61c22f155f77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:28.153464 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543aa5c-717f-40b7-a19d-61c22f155f77 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:28.163435 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 93/179] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:24:27 2022] GET /identity/v3/role_assignments?user.id=93ad24fe28ec40dbbca364cbdb34972d&scope.project.id=791daf53d0364b258d9cfb153ffa3111&role.id=467d49b8a0d3435aaba58464cc8ef85c => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:28.164891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543aa5c-717f-40b7-a19d-61c22f155f77 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:28.278534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6e511-3685-4c37-a8c3-88aa4cce4b11 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:28.313248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:28.327217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:28.330350 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22143c-e547-4a36-828f-1930bdb9d5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:28.331314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22143c-e547-4a36-828f-1930bdb9d5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:28.333519 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22143c-e547-4a36-828f-1930bdb9d5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:28.333519 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22143c-e547-4a36-828f-1930bdb9d5e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:28.333519 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22143c-e547-4a36-828f-1930bdb9d5e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:28.333949 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22143c-e547-4a36-828f-1930bdb9d5e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:28.335724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22143c-e547-4a36-828f-1930bdb9d5e4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:28.341532 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543aa5c-717f-40b7-a19d-61c22f155f77 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:28.357282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 87/180] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:28.357282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22143c-e547-4a36-828f-1930bdb9d5e4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:28.402033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93574218-b3eb-4157-8fb2-f88957245d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:28.403130 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93574218-b3eb-4157-8fb2-f88957245d48 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:28.403506 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93574218-b3eb-4157-8fb2-f88957245d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:28.404042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93574218-b3eb-4157-8fb2-f88957245d48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:28.426747 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93574218-b3eb-4157-8fb2-f88957245d48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:28.441187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-93574218-b3eb-4157-8fb2-f88957245d48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:24:28.441187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:28.441187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:28.441187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:28.441187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:28.441187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:28.441187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:28.441187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:28.441187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:28.441187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:28.441187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:28.441187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:28.441187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:28.441187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:28.441187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:28.441187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:28.442851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:28.442851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:28.442851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:28.442851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:28.442851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:28.442851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:28.442851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:28.442851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:28.442851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:28.442851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:28.442851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:28.442851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:28.442851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:28.442851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:28.442851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:28.442851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:28.445360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:28.445360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:28.445360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:28.445360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:28.445360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:28.445360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:28.445360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:28.445360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:28.445360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:28.445360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:28.445360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:24:28.445360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:28.445360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 94/181] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:24:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:28.446632 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb1be3-ca05-4251-9b0d-9f9e7b20fc46 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:28.446632 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb1be3-ca05-4251-9b0d-9f9e7b20fc46 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:28.446632 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb1be3-ca05-4251-9b0d-9f9e7b20fc46 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:28.449670 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 95/182] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:28.451526 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb1be3-ca05-4251-9b0d-9f9e7b20fc46 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:28.521304 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22143c-e547-4a36-828f-1930bdb9d5e4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:28.565982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2306da3-a357-491d-a9c5-b8b7c7d56d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:28.566906 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2306da3-a357-491d-a9c5-b8b7c7d56d44 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:28.567267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2306da3-a357-491d-a9c5-b8b7c7d56d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:28.567641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2306da3-a357-491d-a9c5-b8b7c7d56d44 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:28.581287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2306da3-a357-491d-a9c5-b8b7c7d56d44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:28.590463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-f2306da3-a357-491d-a9c5-b8b7c7d56d44 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 08:24:28.590463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:28.590463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:28.590463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:28.590463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:28.590463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:28.590463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:28.590463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:28.590463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:28.590463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:28.590463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:28.590463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:28.590463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:28.590463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:28.590463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:28.590463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:28.591710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:28.591710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:28.591710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:28.591710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:28.591710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:28.591710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:28.591710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:28.591710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:28.591710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:28.591710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:28.591710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:28.591710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:28.591710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:28.591710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:28.591710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:28.591710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:28.593631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:28.593631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:28.593631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:28.593631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:28.593631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:28.593631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:28.593631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:28.593631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:28.593631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:28.593631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:28.593631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 08:24:28.593631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:24:28.593631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 88/183] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:28.593631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2306da3-a357-491d-a9c5-b8b7c7d56d44 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:28.609764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb1be3-ca05-4251-9b0d-9f9e7b20fc46 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:28.653685 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5dcfb-688f-47cf-bb34-8ce94cb473bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:28.654651 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5dcfb-688f-47cf-bb34-8ce94cb473bf None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:28.655101 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5dcfb-688f-47cf-bb34-8ce94cb473bf None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:28.655555 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5dcfb-688f-47cf-bb34-8ce94cb473bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:28.660087 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:28.673628 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5dcfb-688f-47cf-bb34-8ce94cb473bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:28.687921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-dcd5dcfb-688f-47cf-bb34-8ce94cb473bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:24:28.687921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:28.687921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:28.687921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:28.687921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:28.687921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:28.687921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:28.687921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:28.687921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:28.687921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:28.687921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:28.687921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:28.687921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:28.687921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:28.687921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:28.687921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:28.689978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:28.689978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:28.689978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:28.689978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:28.689978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:28.689978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:28.689978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:28.689978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:28.689978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:28.689978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:28.689978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:28.689978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:28.689978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:28.689978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:28.689978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:28.689978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:28.691887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:28.691887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:28.691887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:28.691887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:28.691887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:28.691887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:28.691887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:28.691887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:28.691887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:28.691887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:28.691887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:24:28.691887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:28.696027 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 96/184] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:24:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:28.697072 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5dcfb-688f-47cf-bb34-8ce94cb473bf None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:28.725614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:28.744344 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2306da3-a357-491d-a9c5-b8b7c7d56d44 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:28.794642 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dee95f-963e-45a9-b33b-6d67c1e26db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:28.795741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dee95f-963e-45a9-b33b-6d67c1e26db8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:28.796077 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dee95f-963e-45a9-b33b-6d67c1e26db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:28.796508 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dee95f-963e-45a9-b33b-6d67c1e26db8 None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:28.798986 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dee95f-963e-45a9-b33b-6d67c1e26db8 None admin] RBAC: Authorizing `identity:get_project(project_id=791daf53d0364b258d9cfb153ffa3111)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:28.801162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dee95f-963e-45a9-b33b-6d67c1e26db8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:28.804679 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 89/185] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:28 2022] GET /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:28.805984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dee95f-963e-45a9-b33b-6d67c1e26db8 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:28.843208 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5dcfb-688f-47cf-bb34-8ce94cb473bf None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:28.889074 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a91b9-7dec-402a-be3b-b1e7994c34a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:28.890563 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a91b9-7dec-402a-be3b-b1e7994c34a0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:28.891071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a91b9-7dec-402a-be3b-b1e7994c34a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:28.891580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a91b9-7dec-402a-be3b-b1e7994c34a0 None admin] PATH_INFO: `/v3/users/a9e3e8eb9c8c4db0ae0629919a5b1a56` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:28.893856 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a91b9-7dec-402a-be3b-b1e7994c34a0 None admin] RBAC: Authorizing `identity:get_user(user_id=a9e3e8eb9c8c4db0ae0629919a5b1a56)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:28.896814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a91b9-7dec-402a-be3b-b1e7994c34a0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:28.900040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 97/186] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:28 2022] GET /identity/v3/users/a9e3e8eb9c8c4db0ae0629919a5b1a56 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:28.903360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071f818-bd3e-4085-8d20-01fbf231de7e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:28.903896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071f818-bd3e-4085-8d20-01fbf231de7e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:28.904401 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071f818-bd3e-4085-8d20-01fbf231de7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:28.929486 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dee95f-963e-45a9-b33b-6d67c1e26db8 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:28.934815 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a071f818-bd3e-4085-8d20-01fbf231de7e None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:28.936966 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a071f818-bd3e-4085-8d20-01fbf231de7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:28.944469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:28.966887 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9b4cc-08a7-46c7-b5c4-8751a1538d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:28.967575 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9b4cc-08a7-46c7-b5c4-8751a1538d6c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:28.967900 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9b4cc-08a7-46c7-b5c4-8751a1538d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:28.968184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9b4cc-08a7-46c7-b5c4-8751a1538d6c None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:28.968950 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9b4cc-08a7-46c7-b5c4-8751a1538d6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:28.969259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9b4cc-08a7-46c7-b5c4-8751a1538d6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:28.970659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9b4cc-08a7-46c7-b5c4-8751a1538d6c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:28.980731 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 90/187] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:28.981621 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9b4cc-08a7-46c7-b5c4-8751a1538d6c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:29.052952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 98/188] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:29.054404 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071f818-bd3e-4085-8d20-01fbf231de7e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:29.093580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:29.118977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9b4cc-08a7-46c7-b5c4-8751a1538d6c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:29.164172 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ff3e6-5055-4941-8155-aa91b997c7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:29.165040 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ff3e6-5055-4941-8155-aa91b997c7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:29.165406 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ff3e6-5055-4941-8155-aa91b997c7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:29.165864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ff3e6-5055-4941-8155-aa91b997c7a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:29.167700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ff3e6-5055-4941-8155-aa91b997c7a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:29.168076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ff3e6-5055-4941-8155-aa91b997c7a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:29.169604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ff3e6-5055-4941-8155-aa91b997c7a9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:29.181113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 91/189] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:29.183129 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ff3e6-5055-4941-8155-aa91b997c7a9 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:29.213937 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071f818-bd3e-4085-8d20-01fbf231de7e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:29.236782 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:29.261740 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b13a80-2c60-47ef-b32d-18b95b608b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:29.262765 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b13a80-2c60-47ef-b32d-18b95b608b32 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:29.263217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b13a80-2c60-47ef-b32d-18b95b608b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:29.263627 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b13a80-2c60-47ef-b32d-18b95b608b32 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:29.264928 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b13a80-2c60-47ef-b32d-18b95b608b32 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:29.265426 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b13a80-2c60-47ef-b32d-18b95b608b32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:29.267885 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b13a80-2c60-47ef-b32d-18b95b608b32 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:29.282426 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 99/190] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:24:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:29.283839 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b13a80-2c60-47ef-b32d-18b95b608b32 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:29.330667 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ff3e6-5055-4941-8155-aa91b997c7a9 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:29.382601 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b743ffb-4041-49b5-b2d0-51699221a170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:29.383541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b743ffb-4041-49b5-b2d0-51699221a170 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:29.384452 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b743ffb-4041-49b5-b2d0-51699221a170 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:29.384452 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b743ffb-4041-49b5-b2d0-51699221a170 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:29.386340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b743ffb-4041-49b5-b2d0-51699221a170 None admin] RBAC: Adding query filter params (role.id=d2ba35e60f974010a5e7dd5c4bebecf8, scope.project.id=791daf53d0364b258d9cfb153ffa3111, user.id=a9e3e8eb9c8c4db0ae0629919a5b1a56) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:29.386778 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b743ffb-4041-49b5-b2d0-51699221a170 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:29.388490 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b743ffb-4041-49b5-b2d0-51699221a170 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:29.396663 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 92/191] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:24:29 2022] GET /identity/v3/role_assignments?user.id=a9e3e8eb9c8c4db0ae0629919a5b1a56&scope.project.id=791daf53d0364b258d9cfb153ffa3111&role.id=d2ba35e60f974010a5e7dd5c4bebecf8 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:29.397706 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b743ffb-4041-49b5-b2d0-51699221a170 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:29.408280 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:29.448784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b13a80-2c60-47ef-b32d-18b95b608b32 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:29.506149 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00309e96-405e-49c1-88a2-6673914b661b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:29.507331 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00309e96-405e-49c1-88a2-6673914b661b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:29.508036 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00309e96-405e-49c1-88a2-6673914b661b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:29.508593 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00309e96-405e-49c1-88a2-6673914b661b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:29.583405 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b743ffb-4041-49b5-b2d0-51699221a170 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:29.586569 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:29.625666 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00309e96-405e-49c1-88a2-6673914b661b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:29.629461 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00309e96-405e-49c1-88a2-6673914b661b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:29.643723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd5678-1387-4bf1-9737-c0429d9222ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:29.645187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd5678-1387-4bf1-9737-c0429d9222ff None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:29.645663 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd5678-1387-4bf1-9737-c0429d9222ff None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:29.646125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd5678-1387-4bf1-9737-c0429d9222ff None admin] PATH_INFO: `/v3/users/84438f0bf7944e0298f797fed3dd2622` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:29.668306 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd5678-1387-4bf1-9737-c0429d9222ff None admin] RBAC: Authorizing `identity:get_user(user_id=84438f0bf7944e0298f797fed3dd2622)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:29.670815 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd5678-1387-4bf1-9737-c0429d9222ff None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:29.673925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 93/192] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:29 2022] GET /identity/v3/users/84438f0bf7944e0298f797fed3dd2622 => generated 283 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:29.674956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd5678-1387-4bf1-9737-c0429d9222ff None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:29.763847 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:29.803080 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd5678-1387-4bf1-9737-c0429d9222ff None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:29.805126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 100/193] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:24:29.806387 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00309e96-405e-49c1-88a2-6673914b661b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:29.860219 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa1a44-e701-4715-b5a9-a5eb095bf474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:29.861572 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa1a44-e701-4715-b5a9-a5eb095bf474 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:29.862069 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa1a44-e701-4715-b5a9-a5eb095bf474 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:29.862556 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa1a44-e701-4715-b5a9-a5eb095bf474 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:29.864751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa1a44-e701-4715-b5a9-a5eb095bf474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:29.867717 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa1a44-e701-4715-b5a9-a5eb095bf474 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:29.871453 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 94/194] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:24:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:29.872892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa1a44-e701-4715-b5a9-a5eb095bf474 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:29.922605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:29.933531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00309e96-405e-49c1-88a2-6673914b661b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:29.972189 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a3861-07b7-4ac5-9f7e-a6f51ca73b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:29.973079 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a3861-07b7-4ac5-9f7e-a6f51ca73b8c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:29.973355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a3861-07b7-4ac5-9f7e-a6f51ca73b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:29.973631 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a3861-07b7-4ac5-9f7e-a6f51ca73b8c None admin] PATH_INFO: `/v3/users/ab00423871a34113bf9e5853f83e3011` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:29.975005 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a3861-07b7-4ac5-9f7e-a6f51ca73b8c None admin] RBAC: Authorizing `identity:get_user(user_id=ab00423871a34113bf9e5853f83e3011)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:29.976592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a3861-07b7-4ac5-9f7e-a6f51ca73b8c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:29.978294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 101/195] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:29 2022] GET /identity/v3/users/ab00423871a34113bf9e5853f83e3011 => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:29.979728 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a3861-07b7-4ac5-9f7e-a6f51ca73b8c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:30.032214 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:30.057486 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa1a44-e701-4715-b5a9-a5eb095bf474 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:30.092498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a3861-07b7-4ac5-9f7e-a6f51ca73b8c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:30.119413 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4308fb4-f0d5-49f0-8f6f-a5f9c8e41c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:30.120995 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4308fb4-f0d5-49f0-8f6f-a5f9c8e41c7d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:30.121503 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4308fb4-f0d5-49f0-8f6f-a5f9c8e41c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:30.122006 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4308fb4-f0d5-49f0-8f6f-a5f9c8e41c7d None admin] PATH_INFO: `/v3/users/84438f0bf7944e0298f797fed3dd2622` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:30.124271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4308fb4-f0d5-49f0-8f6f-a5f9c8e41c7d None admin] RBAC: Authorizing `identity:get_user(user_id=84438f0bf7944e0298f797fed3dd2622)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:30.126980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4308fb4-f0d5-49f0-8f6f-a5f9c8e41c7d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:30.129342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554fe6b-883c-4262-a2b2-856557022509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:30.129994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554fe6b-883c-4262-a2b2-856557022509 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:30.130184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 95/196] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:29 2022] GET /identity/v3/users/84438f0bf7944e0298f797fed3dd2622 => generated 283 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:30.130314 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554fe6b-883c-4262-a2b2-856557022509 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:30.130544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554fe6b-883c-4262-a2b2-856557022509 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:30.131616 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4308fb4-f0d5-49f0-8f6f-a5f9c8e41c7d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:30.216006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554fe6b-883c-4262-a2b2-856557022509 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:30.217731 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554fe6b-883c-4262-a2b2-856557022509 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:30.247584 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:30.256113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4308fb4-f0d5-49f0-8f6f-a5f9c8e41c7d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:30.282528 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:30.304134 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806658f-e714-4a2b-b549-781264a38800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:30.305206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806658f-e714-4a2b-b549-781264a38800 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:30.305557 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806658f-e714-4a2b-b549-781264a38800 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:30.305907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806658f-e714-4a2b-b549-781264a38800 None admin] PATH_INFO: `/v3/projects/4c4224e757934016a179499c9c15cf75` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:30.309503 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9806658f-e714-4a2b-b549-781264a38800 None admin] RBAC: Authorizing `identity:get_project(project_id=4c4224e757934016a179499c9c15cf75)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:30.311294 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9806658f-e714-4a2b-b549-781264a38800 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:30.313783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 96/197] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:30 2022] GET /identity/v3/projects/4c4224e757934016a179499c9c15cf75 => generated 312 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:30.315402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806658f-e714-4a2b-b549-781264a38800 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:30.384154 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 102/198] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:24:30.384154 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554fe6b-883c-4262-a2b2-856557022509 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:30.468387 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806658f-e714-4a2b-b549-781264a38800 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:30.516772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554fe6b-883c-4262-a2b2-856557022509 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:30.528364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d22bd4-9062-4ef6-b7b8-04b4bd3264bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:30.529071 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d22bd4-9062-4ef6-b7b8-04b4bd3264bc None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:30.529369 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d22bd4-9062-4ef6-b7b8-04b4bd3264bc None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:30.529642 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d22bd4-9062-4ef6-b7b8-04b4bd3264bc None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:30.530390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d22bd4-9062-4ef6-b7b8-04b4bd3264bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:30.531933 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d22bd4-9062-4ef6-b7b8-04b4bd3264bc None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:30.561859 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d082867-15aa-4f15-b591-556043db6934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:30.562454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d082867-15aa-4f15-b591-556043db6934 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:30.562741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d082867-15aa-4f15-b591-556043db6934 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:30.563021 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d082867-15aa-4f15-b591-556043db6934 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:30.589376 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:30.654640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 97/199] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 18 08:24:30 2022] POST /identity/v3/users => generated 297 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:24:30.655735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d22bd4-9062-4ef6-b7b8-04b4bd3264bc None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:30.688607 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:30.692312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d082867-15aa-4f15-b591-556043db6934 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:30.693896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d082867-15aa-4f15-b591-556043db6934 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:30.826981 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 103/200] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:24:30.827698 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d082867-15aa-4f15-b591-556043db6934 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:30.844796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d22bd4-9062-4ef6-b7b8-04b4bd3264bc None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:30.889850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bb563-7817-43ac-baa1-9bd7e2ff3dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:30.890546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bb563-7817-43ac-baa1-9bd7e2ff3dfa None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:30.890826 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bb563-7817-43ac-baa1-9bd7e2ff3dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:30.891101 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bb563-7817-43ac-baa1-9bd7e2ff3dfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:30.904224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:30.998191 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d082867-15aa-4f15-b591-556043db6934 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:31.007819 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5bb563-7817-43ac-baa1-9bd7e2ff3dfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:31.010655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5bb563-7817-43ac-baa1-9bd7e2ff3dfa None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:31.040248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:31.045537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe82af-3f7e-442b-b32e-ee5a2b6f6a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:31.046384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe82af-3f7e-442b-b32e-ee5a2b6f6a76 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:31.046794 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe82af-3f7e-442b-b32e-ee5a2b6f6a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:31.047077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe82af-3f7e-442b-b32e-ee5a2b6f6a76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:31.048165 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe82af-3f7e-442b-b32e-ee5a2b6f6a76 None admin] RBAC: Adding query filter params (role.id=587af95739954600bbf2411974601f74, scope.project.id=4c4224e757934016a179499c9c15cf75, user.id=a9e3e8eb9c8c4db0ae0629919a5b1a56) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:31.048488 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe82af-3f7e-442b-b32e-ee5a2b6f6a76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:31.056794 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe82af-3f7e-442b-b32e-ee5a2b6f6a76 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:31.058405 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 104/201] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:24:30 2022] GET /identity/v3/role_assignments?user.id=a9e3e8eb9c8c4db0ae0629919a5b1a56&scope.project.id=4c4224e757934016a179499c9c15cf75&role.id=587af95739954600bbf2411974601f74 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:31.059381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe82af-3f7e-442b-b32e-ee5a2b6f6a76 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:31.262400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 98/202] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:24:31.263116 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bb563-7817-43ac-baa1-9bd7e2ff3dfa None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:31.264716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe82af-3f7e-442b-b32e-ee5a2b6f6a76 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:31.279208 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:31.323756 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107d721-b127-476a-8fec-5aad779d8802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:31.326305 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107d721-b127-476a-8fec-5aad779d8802 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:31.327208 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107d721-b127-476a-8fec-5aad779d8802 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:31.327772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107d721-b127-476a-8fec-5aad779d8802 None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:31.331404 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107d721-b127-476a-8fec-5aad779d8802 None admin] RBAC: Authorizing `identity:get_project(project_id=21519c13797a4d5b8858f4eed4d738f8)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:31.334852 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107d721-b127-476a-8fec-5aad779d8802 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:31.339831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 105/203] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:31 2022] GET /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8 => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:31.341402 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107d721-b127-476a-8fec-5aad779d8802 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:31.469187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bb563-7817-43ac-baa1-9bd7e2ff3dfa None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:31.499055 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:31.519684 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107d721-b127-476a-8fec-5aad779d8802 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:31.519871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae703aef-5489-4bb9-8787-b98834c45978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:31.519871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae703aef-5489-4bb9-8787-b98834c45978 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:31.519871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae703aef-5489-4bb9-8787-b98834c45978 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:31.519871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae703aef-5489-4bb9-8787-b98834c45978 None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:31.520155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae703aef-5489-4bb9-8787-b98834c45978 None admin] RBAC: Authorizing `identity:get_project(project_id=791daf53d0364b258d9cfb153ffa3111)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:31.520602 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae703aef-5489-4bb9-8787-b98834c45978 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:31.522724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 99/204] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:31 2022] GET /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:31.525006 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae703aef-5489-4bb9-8787-b98834c45978 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:31.572069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a820edd-e8cd-4d56-87be-96bf6cbe51f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:31.573868 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a820edd-e8cd-4d56-87be-96bf6cbe51f3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:31.574391 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a820edd-e8cd-4d56-87be-96bf6cbe51f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:31.574868 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a820edd-e8cd-4d56-87be-96bf6cbe51f3 None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:31.576763 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a820edd-e8cd-4d56-87be-96bf6cbe51f3 None admin] RBAC: Authorizing `identity:get_project(project_id=21519c13797a4d5b8858f4eed4d738f8)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:31.580052 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a820edd-e8cd-4d56-87be-96bf6cbe51f3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:31.582431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 106/205] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:31 2022] GET /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:31.583749 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a820edd-e8cd-4d56-87be-96bf6cbe51f3 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:31.656041 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:31.671592 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae703aef-5489-4bb9-8787-b98834c45978 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:31.720216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4efcb8-7b0f-4243-bc86-ee12a2fbadc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:31.720898 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4efcb8-7b0f-4243-bc86-ee12a2fbadc2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:31.721187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4efcb8-7b0f-4243-bc86-ee12a2fbadc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:31.721469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4efcb8-7b0f-4243-bc86-ee12a2fbadc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:31.722065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a820edd-e8cd-4d56-87be-96bf6cbe51f3 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:31.722523 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4efcb8-7b0f-4243-bc86-ee12a2fbadc2 None admin] RBAC: Adding query filter params (role.id=467d49b8a0d3435aaba58464cc8ef85c, scope.project.id=21519c13797a4d5b8858f4eed4d738f8, user.id=84438f0bf7944e0298f797fed3dd2622) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:31.722687 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4efcb8-7b0f-4243-bc86-ee12a2fbadc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:31.724184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4efcb8-7b0f-4243-bc86-ee12a2fbadc2 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:31.733420 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 100/206] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:24:31 2022] GET /identity/v3/role_assignments?user.id=84438f0bf7944e0298f797fed3dd2622&scope.project.id=21519c13797a4d5b8858f4eed4d738f8&role.id=467d49b8a0d3435aaba58464cc8ef85c => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:31.734215 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4efcb8-7b0f-4243-bc86-ee12a2fbadc2 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:31.777955 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7991e-bb68-432c-87e4-a37bc7f6a61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:31.778553 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7991e-bb68-432c-87e4-a37bc7f6a61e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:31.779803 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7991e-bb68-432c-87e4-a37bc7f6a61e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:31.784246 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7991e-bb68-432c-87e4-a37bc7f6a61e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:31.786072 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e7991e-bb68-432c-87e4-a37bc7f6a61e None admin] RBAC: Adding query filter params (role.id=24179486bd6a4eb985a83d47ca5c2b4f, scope.project.id=791daf53d0364b258d9cfb153ffa3111, user.id=ab00423871a34113bf9e5853f83e3011) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:31.786476 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e7991e-bb68-432c-87e4-a37bc7f6a61e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:31.788939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e7991e-bb68-432c-87e4-a37bc7f6a61e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:31.798941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 107/207] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:24:31 2022] GET /identity/v3/role_assignments?user.id=ab00423871a34113bf9e5853f83e3011&scope.project.id=791daf53d0364b258d9cfb153ffa3111&role.id=24179486bd6a4eb985a83d47ca5c2b4f => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:31.811010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:31.937487 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4efcb8-7b0f-4243-bc86-ee12a2fbadc2 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:32.026683 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b3b90-6d7b-44c0-a291-0ce641864dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:32.028495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b3b90-6d7b-44c0-a291-0ce641864dfa None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:32.029374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b3b90-6d7b-44c0-a291-0ce641864dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:32.030161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b3b90-6d7b-44c0-a291-0ce641864dfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:32.032447 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b3b90-6d7b-44c0-a291-0ce641864dfa None admin] RBAC: Adding query filter params (role.id=d2ba35e60f974010a5e7dd5c4bebecf8, scope.project.id=21519c13797a4d5b8858f4eed4d738f8, user.id=84438f0bf7944e0298f797fed3dd2622) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:32.033471 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b3b90-6d7b-44c0-a291-0ce641864dfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:32.036852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b3b90-6d7b-44c0-a291-0ce641864dfa None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:32.053407 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 101/208] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:24:31 2022] GET /identity/v3/role_assignments?user.id=84438f0bf7944e0298f797fed3dd2622&scope.project.id=21519c13797a4d5b8858f4eed4d738f8&role.id=d2ba35e60f974010a5e7dd5c4bebecf8 => generated 259 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:33.198401 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba27193-53bc-4973-a503-bb8380489147 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:33.199189 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba27193-53bc-4973-a503-bb8380489147 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:33.199189 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba27193-53bc-4973-a503-bb8380489147 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:33.200464 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 108/209] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:33.206510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6617b-0ca2-4cc0-9f6f-25ef903efe8d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:33.207346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6617b-0ca2-4cc0-9f6f-25ef903efe8d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:33.207346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6617b-0ca2-4cc0-9f6f-25ef903efe8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:33.227374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0ea6617b-0ca2-4cc0-9f6f-25ef903efe8d None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:33.229619 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0ea6617b-0ca2-4cc0-9f6f-25ef903efe8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:33.319335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 102/210] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:24:33.362819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba27193-53bc-4973-a503-bb8380489147 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:33.504531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba27193-53bc-4973-a503-bb8380489147 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:33.514893 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:33.543811 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f8beb-3ba4-4777-babf-48adeb62eda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:33.544634 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f8beb-3ba4-4777-babf-48adeb62eda3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:33.544921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f8beb-3ba4-4777-babf-48adeb62eda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:33.545199 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f8beb-3ba4-4777-babf-48adeb62eda3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:33.546494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f8beb-3ba4-4777-babf-48adeb62eda3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:33.548062 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f8beb-3ba4-4777-babf-48adeb62eda3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:33.550005 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 109/211] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:24:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:33.555741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6617b-0ca2-4cc0-9f6f-25ef903efe8d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:33.653517 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:33.714437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62622e3c-3bae-4d0f-abb0-d9c77e8b77d5 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:33.714962 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62622e3c-3bae-4d0f-abb0-d9c77e8b77d5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:33.715385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62622e3c-3bae-4d0f-abb0-d9c77e8b77d5 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:33.721147 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 110/212] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:33.723150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7d76e-f109-41b0-9382-30f3525f8b44 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:33.723719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7d76e-f109-41b0-9382-30f3525f8b44 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:33.724395 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7d76e-f109-41b0-9382-30f3525f8b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:33.748137 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-62f7d76e-f109-41b0-9382-30f3525f8b44 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:33.749886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-62f7d76e-f109-41b0-9382-30f3525f8b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:33.761093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6617b-0ca2-4cc0-9f6f-25ef903efe8d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:33.828100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef05ca-9949-4dd2-803b-0e68e6ee8ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:33.829221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef05ca-9949-4dd2-803b-0e68e6ee8ba4 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:33.829703 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef05ca-9949-4dd2-803b-0e68e6ee8ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:33.830169 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef05ca-9949-4dd2-803b-0e68e6ee8ba4 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:33.831568 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef05ca-9949-4dd2-803b-0e68e6ee8ba4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:33.834488 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef05ca-9949-4dd2-803b-0e68e6ee8ba4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:33.844911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 111/213] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:24:33.872823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7d76e-f109-41b0-9382-30f3525f8b44 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:33.935623 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 103/214] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 18 08:24:33 2022] POST /identity/v3/users => generated 297 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:24:34.055245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7d76e-f109-41b0-9382-30f3525f8b44 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:34.069142 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:34.119332 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f19891-d8d5-4a73-94e2-762d1d1607a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:34.120943 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f19891-d8d5-4a73-94e2-762d1d1607a8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:34.121421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f19891-d8d5-4a73-94e2-762d1d1607a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:34.121889 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f19891-d8d5-4a73-94e2-762d1d1607a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:34.142737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f19891-d8d5-4a73-94e2-762d1d1607a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:34.158489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-d2f19891-d8d5-4a73-94e2-762d1d1607a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:34.158489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:34.158489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:34.158489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:34.158489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:34.158489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:34.158489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:34.158489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:34.158489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:34.158489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:34.158489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:34.158489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:34.158489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:34.158489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:34.158489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:34.158489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:34.160273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:34.160273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:34.160273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:34.160273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:34.160273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:34.160273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:34.160273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:34.160273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:34.160273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:34.160273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:34.160273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:34.160273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:34.160273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:34.160273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:34.160273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:34.160273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:34.161805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:34.161805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:34.161805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:34.161805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:34.161805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:34.161805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:34.161805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:34.161805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:34.161805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:34.161805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:34.161805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:34.161805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:34.167841 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 112/215] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:24:34.170576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef05ca-9949-4dd2-803b-0e68e6ee8ba4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:34.353983 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef05ca-9949-4dd2-803b-0e68e6ee8ba4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:34.356703 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:34.421329 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efc639-c1d1-462d-b8fb-b873fac8c6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:34.422420 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efc639-c1d1-462d-b8fb-b873fac8c6b2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:34.422903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efc639-c1d1-462d-b8fb-b873fac8c6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:34.423378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efc639-c1d1-462d-b8fb-b873fac8c6b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:34.424765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efc639-c1d1-462d-b8fb-b873fac8c6b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:34.425283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efc639-c1d1-462d-b8fb-b873fac8c6b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:34.427861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efc639-c1d1-462d-b8fb-b873fac8c6b2 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:34.433615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a1b75-a18d-416d-9d19-644a3a63168c None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:34.434468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a1b75-a18d-416d-9d19-644a3a63168c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:34.434780 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a1b75-a18d-416d-9d19-644a3a63168c None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:34.437095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 113/216] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:34.449297 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46a23f-fe26-462d-8b25-8a555c4c522d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:34.449567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46a23f-fe26-462d-8b25-8a555c4c522d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:34.449860 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46a23f-fe26-462d-8b25-8a555c4c522d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:34.452545 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 104/217] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:34.463926 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efc639-c1d1-462d-b8fb-b873fac8c6b2 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:34.491623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3f46a23f-fe26-462d-8b25-8a555c4c522d None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:34.494037 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3f46a23f-fe26-462d-8b25-8a555c4c522d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:34.637549 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 114/218] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:34.685884 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46a23f-fe26-462d-8b25-8a555c4c522d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:34.687517 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efc639-c1d1-462d-b8fb-b873fac8c6b2 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:34.790429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2f2fa-a907-447f-8657-6e1754587966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:34.792100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2f2fa-a907-447f-8657-6e1754587966 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:34.792609 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2f2fa-a907-447f-8657-6e1754587966 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:34.793108 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2f2fa-a907-447f-8657-6e1754587966 None admin] PATH_INFO: `/v3/users/d1243e4757d844928dd2cd699318de60` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:34.857915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2f2fa-a907-447f-8657-6e1754587966 None admin] RBAC: Authorizing `identity:get_user(user_id=d1243e4757d844928dd2cd699318de60)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:34.860803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2f2fa-a907-447f-8657-6e1754587966 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:34.864112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 105/219] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:34 2022] GET /identity/v3/users/d1243e4757d844928dd2cd699318de60 => generated 297 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:34.874116 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2f2fa-a907-447f-8657-6e1754587966 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:34.909129 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:34.922192 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46a23f-fe26-462d-8b25-8a555c4c522d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:34.970716 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:34.976364 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6ceb4-f70d-4e5c-a1ad-d0f7126c547a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:34.977628 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6ceb4-f70d-4e5c-a1ad-d0f7126c547a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:34.978116 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6ceb4-f70d-4e5c-a1ad-d0f7126c547a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:34.978628 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6ceb4-f70d-4e5c-a1ad-d0f7126c547a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:34.991436 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6ceb4-f70d-4e5c-a1ad-d0f7126c547a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:35.000735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-77e6ceb4-f70d-4e5c-a1ad-d0f7126c547a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:24:35.000735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:35.000735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:35.000735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:35.000735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:35.000735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:35.000735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:35.000735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:35.000735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:35.000735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:35.000735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:35.000735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:35.000735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:35.000735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:35.000735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:35.000735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:35.001718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:35.001718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:35.001718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:35.001718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:35.001718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:35.001718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:35.001718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:35.001718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:35.001718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:35.001718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:35.001718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:35.001718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:35.001718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:35.001718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:35.001718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:35.001718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:35.002517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:35.002517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:35.002517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:35.002517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:35.002517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:35.002517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:35.002517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:35.002517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:35.002517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:35.002517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:35.002517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:24:35.002517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:35.005442 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 115/220] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:24:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:35.016165 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6ceb4-f70d-4e5c-a1ad-d0f7126c547a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:35.091961 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2f2fa-a907-447f-8657-6e1754587966 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:35.130925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ad0e4-4d65-4ab6-9dba-b816572b82c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:35.131615 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ad0e4-4d65-4ab6-9dba-b816572b82c9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:35.132053 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ad0e4-4d65-4ab6-9dba-b816572b82c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:35.132340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ad0e4-4d65-4ab6-9dba-b816572b82c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:35.218033 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ad0e4-4d65-4ab6-9dba-b816572b82c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:35.220597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6ceb4-f70d-4e5c-a1ad-d0f7126c547a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:35.220967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ad0e4-4d65-4ab6-9dba-b816572b82c9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:35.281133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:35.301296 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc31908-c6a7-4ded-bdae-6e478b030f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:35.302506 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc31908-c6a7-4ded-bdae-6e478b030f69 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:35.303029 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc31908-c6a7-4ded-bdae-6e478b030f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:35.304405 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc31908-c6a7-4ded-bdae-6e478b030f69 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:35.305861 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc31908-c6a7-4ded-bdae-6e478b030f69 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:35.306474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc31908-c6a7-4ded-bdae-6e478b030f69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:35.309197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc31908-c6a7-4ded-bdae-6e478b030f69 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:35.327734 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 116/221] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:24:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:35.330240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6301f-7f9f-473f-b94c-acd12c352812 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:35.330743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6301f-7f9f-473f-b94c-acd12c352812 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:35.331234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6301f-7f9f-473f-b94c-acd12c352812 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:35.333819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 117/222] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:35.335451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6301f-7f9f-473f-b94c-acd12c352812 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:35.377612 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:35.473653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 106/223] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:24:35.483391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0577e62-3c1a-4c80-af88-120b5dafe1b4 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:35.484338 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0577e62-3c1a-4c80-af88-120b5dafe1b4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:35.485159 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0577e62-3c1a-4c80-af88-120b5dafe1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:35.524652 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-b0577e62-3c1a-4c80-af88-120b5dafe1b4 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:35.527642 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b0577e62-3c1a-4c80-af88-120b5dafe1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:35.565408 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6301f-7f9f-473f-b94c-acd12c352812 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:35.669589 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 107/224] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:35.671709 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0577e62-3c1a-4c80-af88-120b5dafe1b4 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:35.684776 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59830c-704d-47f8-a731-3dd52d7e5335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:35.686327 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59830c-704d-47f8-a731-3dd52d7e5335 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:35.686840 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59830c-704d-47f8-a731-3dd52d7e5335 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:35.687345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59830c-704d-47f8-a731-3dd52d7e5335 None admin] PATH_INFO: `/v3/users/84438f0bf7944e0298f797fed3dd2622` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:35.689889 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb59830c-704d-47f8-a731-3dd52d7e5335 None admin] RBAC: Authorizing `identity:get_user(user_id=84438f0bf7944e0298f797fed3dd2622)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:35.693933 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb59830c-704d-47f8-a731-3dd52d7e5335 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:35.698833 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 118/225] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:35 2022] GET /identity/v3/users/84438f0bf7944e0298f797fed3dd2622 => generated 283 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:35.710051 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59830c-704d-47f8-a731-3dd52d7e5335 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:35.800349 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:35.901266 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0577e62-3c1a-4c80-af88-120b5dafe1b4 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:35.936207 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:35.948476 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59830c-704d-47f8-a731-3dd52d7e5335 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:35.994953 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293e484-ec00-4110-a07b-2455008c7fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:35.998097 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293e484-ec00-4110-a07b-2455008c7fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:35.999922 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293e484-ec00-4110-a07b-2455008c7fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:36.001129 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293e484-ec00-4110-a07b-2455008c7fa4 None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:36.004929 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293e484-ec00-4110-a07b-2455008c7fa4 None admin] RBAC: Authorizing `identity:get_project(project_id=21519c13797a4d5b8858f4eed4d738f8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:36.009086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293e484-ec00-4110-a07b-2455008c7fa4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:36.015459 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 108/226] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:35 2022] GET /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8 => generated 302 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:36.017333 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293e484-ec00-4110-a07b-2455008c7fa4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:36.028188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d1e2a-8f07-4739-97a8-8893b227d3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:36.029730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d1e2a-8f07-4739-97a8-8893b227d3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:36.030186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d1e2a-8f07-4739-97a8-8893b227d3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:36.030688 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d1e2a-8f07-4739-97a8-8893b227d3d0 None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:36.033367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d1e2a-8f07-4739-97a8-8893b227d3d0 None admin] RBAC: Authorizing `identity:get_project(project_id=21519c13797a4d5b8858f4eed4d738f8)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:36.036192 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d1e2a-8f07-4739-97a8-8893b227d3d0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:36.039722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 119/227] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:35 2022] GET /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8 => generated 302 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:36.041611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d1e2a-8f07-4739-97a8-8893b227d3d0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:36.239643 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293e484-ec00-4110-a07b-2455008c7fa4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:36.253960 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d1e2a-8f07-4739-97a8-8893b227d3d0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:36.290256 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:36.318989 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6957a-4b94-478b-a731-97026ab6eb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:36.319746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d84b0-c673-4c44-ae87-8b6ea29794d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:36.321080 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d84b0-c673-4c44-ae87-8b6ea29794d3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:36.321488 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d84b0-c673-4c44-ae87-8b6ea29794d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:36.321998 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d84b0-c673-4c44-ae87-8b6ea29794d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:36.322346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6957a-4b94-478b-a731-97026ab6eb21 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:36.323103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6957a-4b94-478b-a731-97026ab6eb21 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:36.323775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d84b0-c673-4c44-ae87-8b6ea29794d3 None admin] RBAC: Adding query filter params (role.id=587af95739954600bbf2411974601f74, scope.project.id=21519c13797a4d5b8858f4eed4d738f8, user.id=d1243e4757d844928dd2cd699318de60) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:36.324156 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6957a-4b94-478b-a731-97026ab6eb21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:36.324328 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d84b0-c673-4c44-ae87-8b6ea29794d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:36.327194 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d84b0-c673-4c44-ae87-8b6ea29794d3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:36.338967 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 120/228] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:24:36 2022] GET /identity/v3/role_assignments?user.id=d1243e4757d844928dd2cd699318de60&scope.project.id=21519c13797a4d5b8858f4eed4d738f8&role.id=587af95739954600bbf2411974601f74 => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:36.346424 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d84b0-c673-4c44-ae87-8b6ea29794d3 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:36.348154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6957a-4b94-478b-a731-97026ab6eb21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:36.364157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-b5c6957a-4b94-478b-a731-97026ab6eb21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:24:36.364157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:36.364157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:36.364157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:36.364157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:36.364157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:36.364157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:36.364157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:36.364157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:36.364157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:36.364157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:36.364157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:36.364157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:36.364157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:36.364157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:36.364157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:36.365861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:36.365861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:36.365861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:36.365861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:36.365861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:36.365861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:36.365861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:36.365861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:36.365861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:36.365861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:36.365861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:36.365861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:36.365861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:36.365861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:36.365861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:36.365861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:36.367641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:36.367641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:36.367641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:36.367641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:36.367641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:36.367641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:36.367641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:36.367641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:36.367641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:36.367641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:36.367641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:24:36.367641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:24:36.372787 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 109/229] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:24:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:36.382079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6957a-4b94-478b-a731-97026ab6eb21 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:36.402546 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:36.602155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d84b0-c673-4c44-ae87-8b6ea29794d3 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:36.686052 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446adc59-86a9-4c1d-af33-a1dc8dff3fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:36.687629 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446adc59-86a9-4c1d-af33-a1dc8dff3fa9 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:36.688190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446adc59-86a9-4c1d-af33-a1dc8dff3fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:36.688686 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446adc59-86a9-4c1d-af33-a1dc8dff3fa9 None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8/users/84438f0bf7944e0298f797fed3dd2622/roles/d2ba35e60f974010a5e7dd5c4bebecf8` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:36.692912 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446adc59-86a9-4c1d-af33-a1dc8dff3fa9 None admin] RBAC: Authorizing `identity:create_grant(project_id=21519c13797a4d5b8858f4eed4d738f8, user_id=84438f0bf7944e0298f797fed3dd2622, role_id=d2ba35e60f974010a5e7dd5c4bebecf8)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:36.696036 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446adc59-86a9-4c1d-af33-a1dc8dff3fa9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:36.733679 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6957a-4b94-478b-a731-97026ab6eb21 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:36.794678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 121/230] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:24:36 2022] PUT /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8/users/84438f0bf7944e0298f797fed3dd2622/roles/d2ba35e60f974010a5e7dd5c4bebecf8 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:24:36.798388 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956a3c6-06a8-48bd-a532-853abbe51d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:36.799574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956a3c6-06a8-48bd-a532-853abbe51d73 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:36.800185 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956a3c6-06a8-48bd-a532-853abbe51d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:36.800680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956a3c6-06a8-48bd-a532-853abbe51d73 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:36.802137 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956a3c6-06a8-48bd-a532-853abbe51d73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:36.802757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956a3c6-06a8-48bd-a532-853abbe51d73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:36.805805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956a3c6-06a8-48bd-a532-853abbe51d73 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:36.823603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 110/231] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:36.833530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446adc59-86a9-4c1d-af33-a1dc8dff3fa9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:37.009095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:37.024011 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3ff08-9847-4da4-a543-4e3616c248d2 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:37.024777 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3ff08-9847-4da4-a543-4e3616c248d2 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:37.025251 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3ff08-9847-4da4-a543-4e3616c248d2 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:37.032062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 111/232] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:37.033802 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cba63f-61f3-4fe2-8bc9-2aa76b1e1d71 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:37.034321 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cba63f-61f3-4fe2-8bc9-2aa76b1e1d71 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:37.034841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cba63f-61f3-4fe2-8bc9-2aa76b1e1d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:37.056594 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446adc59-86a9-4c1d-af33-a1dc8dff3fa9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:37.084642 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a7cba63f-61f3-4fe2-8bc9-2aa76b1e1d71 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:37.087282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a7cba63f-61f3-4fe2-8bc9-2aa76b1e1d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:37.108144 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da301f0-82f3-4fd8-91f4-1185d933bc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:37.109050 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da301f0-82f3-4fd8-91f4-1185d933bc75 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:37.109335 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da301f0-82f3-4fd8-91f4-1185d933bc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:37.109629 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da301f0-82f3-4fd8-91f4-1185d933bc75 None admin] PATH_INFO: `/v3/users/84438f0bf7944e0298f797fed3dd2622` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:37.110928 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da301f0-82f3-4fd8-91f4-1185d933bc75 None admin] RBAC: Authorizing `identity:get_user(user_id=84438f0bf7944e0298f797fed3dd2622)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:37.112504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da301f0-82f3-4fd8-91f4-1185d933bc75 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:37.114520 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 122/233] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:36 2022] GET /identity/v3/users/84438f0bf7944e0298f797fed3dd2622 => generated 283 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:37.115552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d85e2-d5d7-4f5a-8255-14c03920da4a None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:37.115888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d85e2-d5d7-4f5a-8255-14c03920da4a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:37.116149 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d85e2-d5d7-4f5a-8255-14c03920da4a None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:37.117734 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 123/234] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:37.124528 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2764cad-16ba-4040-924c-adc47da7806e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:37.124921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2764cad-16ba-4040-924c-adc47da7806e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:37.125282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2764cad-16ba-4040-924c-adc47da7806e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:37.149743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e2764cad-16ba-4040-924c-adc47da7806e None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:37.151203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e2764cad-16ba-4040-924c-adc47da7806e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:37.183577 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:37.231895 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 112/235] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:37.232773 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cba63f-61f3-4fe2-8bc9-2aa76b1e1d71 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:37.296339 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 124/236] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:24:37.298252 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2764cad-16ba-4040-924c-adc47da7806e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:37.404719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cba63f-61f3-4fe2-8bc9-2aa76b1e1d71 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:37.455874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:37.477144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25488f-07cf-4f09-9dd6-f71918a0d05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:37.478177 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25488f-07cf-4f09-9dd6-f71918a0d05c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:37.478988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25488f-07cf-4f09-9dd6-f71918a0d05c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:37.479272 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25488f-07cf-4f09-9dd6-f71918a0d05c None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:37.489795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25488f-07cf-4f09-9dd6-f71918a0d05c None admin] RBAC: Authorizing `identity:get_project(project_id=21519c13797a4d5b8858f4eed4d738f8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:37.489795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25488f-07cf-4f09-9dd6-f71918a0d05c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:37.489795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 113/237] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:37 2022] GET /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:37.489795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25488f-07cf-4f09-9dd6-f71918a0d05c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:37.501023 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2764cad-16ba-4040-924c-adc47da7806e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:37.540665 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:37.570998 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491d504-8f59-41fe-8bc0-e59c047d10c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:37.571754 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491d504-8f59-41fe-8bc0-e59c047d10c6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:37.572036 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491d504-8f59-41fe-8bc0-e59c047d10c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:37.573412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491d504-8f59-41fe-8bc0-e59c047d10c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:37.575528 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491d504-8f59-41fe-8bc0-e59c047d10c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:37.577672 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491d504-8f59-41fe-8bc0-e59c047d10c6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:37.583777 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 125/238] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:24:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:37.585795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491d504-8f59-41fe-8bc0-e59c047d10c6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:37.658313 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25488f-07cf-4f09-9dd6-f71918a0d05c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:37.724224 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8c0ee-62fd-4d50-b3ce-8c20d5603446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:37.725374 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491d504-8f59-41fe-8bc0-e59c047d10c6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:37.725812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8c0ee-62fd-4d50-b3ce-8c20d5603446 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:37.726031 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8c0ee-62fd-4d50-b3ce-8c20d5603446 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:37.726522 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8c0ee-62fd-4d50-b3ce-8c20d5603446 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:37.749038 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8c0ee-62fd-4d50-b3ce-8c20d5603446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:37.765221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-66c8c0ee-62fd-4d50-b3ce-8c20d5603446 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 08:24:37.765221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:37.765221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:37.765221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:37.765221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:37.765221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:37.765221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:37.765221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:37.765221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:37.765221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:37.765221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:37.765221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:37.765221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:37.765221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:37.765221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:37.765221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:37.766176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:37.766176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:37.766176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:37.766176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:37.766176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:37.766176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:37.766176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:37.766176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:37.766176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:37.766176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:37.766176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:37.766176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:37.766176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:37.766176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:37.766176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:37.766176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:37.766984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:37.766984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:37.766984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:37.766984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:37.766984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:37.766984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:37.766984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:37.766984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:37.766984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:37.766984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:37.766984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 08:24:37.766984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:24:37.773880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 114/239] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:37.776265 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8c0ee-62fd-4d50-b3ce-8c20d5603446 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:37.777969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6bb059-564b-4369-b20e-d71e548421b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:37.779161 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bb059-564b-4369-b20e-d71e548421b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:37.779661 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bb059-564b-4369-b20e-d71e548421b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:37.780163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bb059-564b-4369-b20e-d71e548421b0 None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8/users/84438f0bf7944e0298f797fed3dd2622/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:37.784677 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6bb059-564b-4369-b20e-d71e548421b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=21519c13797a4d5b8858f4eed4d738f8, user_id=84438f0bf7944e0298f797fed3dd2622, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:37.787531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6bb059-564b-4369-b20e-d71e548421b0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:37.855532 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 126/240] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:24:37 2022] PUT /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8/users/84438f0bf7944e0298f797fed3dd2622/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:24:37.856872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6bb059-564b-4369-b20e-d71e548421b0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:37.862096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:38.074636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:38.084565 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8c0ee-62fd-4d50-b3ce-8c20d5603446 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:38.162016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6bb059-564b-4369-b20e-d71e548421b0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:38.191893 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c08966-c074-4a53-9288-a1d9386f41fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:38.191893 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c08966-c074-4a53-9288-a1d9386f41fc None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:38.191893 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c08966-c074-4a53-9288-a1d9386f41fc None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:38.191893 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c08966-c074-4a53-9288-a1d9386f41fc None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:38.192597 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c08966-c074-4a53-9288-a1d9386f41fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:38.192597 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c08966-c074-4a53-9288-a1d9386f41fc None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:38.252632 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ded17-f0e2-43fd-b9d5-b5b078bd0400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:38.254120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ded17-f0e2-43fd-b9d5-b5b078bd0400 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:38.254823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ded17-f0e2-43fd-b9d5-b5b078bd0400 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:38.255469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ded17-f0e2-43fd-b9d5-b5b078bd0400 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:38.257111 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ded17-f0e2-43fd-b9d5-b5b078bd0400 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:38.257866 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ded17-f0e2-43fd-b9d5-b5b078bd0400 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:38.262414 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ded17-f0e2-43fd-b9d5-b5b078bd0400 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:38.282125 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 127/241] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:38.295708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ded17-f0e2-43fd-b9d5-b5b078bd0400 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:38.355082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 115/242] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 18 08:24:37 2022] POST /identity/v3/users => generated 293 bytes in 580 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:24:38.437331 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:38.517216 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ded17-f0e2-43fd-b9d5-b5b078bd0400 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:38.593490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300c87a-fb2e-470c-a1f6-6b66865aceca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:38.595479 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300c87a-fb2e-470c-a1f6-6b66865aceca None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:38.596160 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300c87a-fb2e-470c-a1f6-6b66865aceca None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:38.596767 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300c87a-fb2e-470c-a1f6-6b66865aceca None admin] PATH_INFO: `/v3/users/c6d48aa625764ab9b2d848981827f091` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:38.626979 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300c87a-fb2e-470c-a1f6-6b66865aceca None admin] RBAC: Authorizing `identity:get_user(user_id=c6d48aa625764ab9b2d848981827f091)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:38.629949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300c87a-fb2e-470c-a1f6-6b66865aceca None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:38.633173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 128/243] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:38 2022] GET /identity/v3/users/c6d48aa625764ab9b2d848981827f091 => generated 297 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:38.642325 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c08966-c074-4a53-9288-a1d9386f41fc None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:38.906360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c08966-c074-4a53-9288-a1d9386f41fc None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:38.988439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24adc5-3610-405c-8699-4ff2e2f0a641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:38.989962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24adc5-3610-405c-8699-4ff2e2f0a641 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:38.990632 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24adc5-3610-405c-8699-4ff2e2f0a641 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:38.991319 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24adc5-3610-405c-8699-4ff2e2f0a641 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:39.052510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:39.149992 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24adc5-3610-405c-8699-4ff2e2f0a641 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:39.153788 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24adc5-3610-405c-8699-4ff2e2f0a641 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:39.200108 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1455e6-14c2-48f7-854f-8af92092a29e None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:39.200919 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1455e6-14c2-48f7-854f-8af92092a29e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:39.200919 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1455e6-14c2-48f7-854f-8af92092a29e None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:39.202446 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 129/244] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:39.209002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8137b-93fa-4f57-b1d7-2cca24937f3b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:39.209277 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8137b-93fa-4f57-b1d7-2cca24937f3b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:39.209606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8137b-93fa-4f57-b1d7-2cca24937f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:39.232960 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d0f8137b-93fa-4f57-b1d7-2cca24937f3b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:39.233247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d0f8137b-93fa-4f57-b1d7-2cca24937f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:39.361770 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 116/245] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:24:39.373838 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24adc5-3610-405c-8699-4ff2e2f0a641 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:39.408771 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 130/246] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:39.444910 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8137b-93fa-4f57-b1d7-2cca24937f3b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:39.540077 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:39.614760 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:39.652201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24adc5-3610-405c-8699-4ff2e2f0a641 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:39.729379 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8137b-93fa-4f57-b1d7-2cca24937f3b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:39.734584 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473d099-9b73-4d76-b48c-19578c9b0f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:39.737931 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473d099-9b73-4d76-b48c-19578c9b0f5b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:39.738783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473d099-9b73-4d76-b48c-19578c9b0f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:39.739476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473d099-9b73-4d76-b48c-19578c9b0f5b None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:39.742315 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473d099-9b73-4d76-b48c-19578c9b0f5b None admin] RBAC: Authorizing `identity:get_project(project_id=21519c13797a4d5b8858f4eed4d738f8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:39.746748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473d099-9b73-4d76-b48c-19578c9b0f5b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:39.750885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 117/247] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:39 2022] GET /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8 => generated 302 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:39.754313 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e01a5a-569d-421e-9cf2-8ef48de4fdfd None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:39.755760 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e01a5a-569d-421e-9cf2-8ef48de4fdfd None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:39.763707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e01a5a-569d-421e-9cf2-8ef48de4fdfd None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:39.772024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 118/248] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:39 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:39.773550 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e01a5a-569d-421e-9cf2-8ef48de4fdfd None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:39.827149 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c678ebb-b86f-4c5d-a562-aa7c89e58747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:39.828602 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c678ebb-b86f-4c5d-a562-aa7c89e58747 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:39.829075 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c678ebb-b86f-4c5d-a562-aa7c89e58747 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:39.829568 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c678ebb-b86f-4c5d-a562-aa7c89e58747 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:39.870500 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c678ebb-b86f-4c5d-a562-aa7c89e58747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:39.904422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-1c678ebb-b86f-4c5d-a562-aa7c89e58747 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:39.904422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:39.904422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:39.904422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:39.904422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:39.904422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:39.904422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:39.904422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:39.904422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:39.904422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:39.904422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:39.904422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:39.904422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:39.904422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:39.904422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:39.904422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:39.906234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:39.906234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:39.906234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:39.906234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:39.906234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:39.906234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:39.906234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:39.906234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:39.906234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:39.906234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:39.906234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:39.906234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:39.906234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:39.906234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:39.906234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:39.906234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:39.908615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:39.908615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:39.908615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:39.908615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:39.908615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:39.908615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:39.908615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:39.908615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:39.908615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:39.908615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:39.908615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:39.908615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:39.908615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 131/249] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 463 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:39.909940 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c62b3c-4234-4ded-892a-b044a9d50279 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:39.909940 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c62b3c-4234-4ded-892a-b044a9d50279 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:39.910182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c62b3c-4234-4ded-892a-b044a9d50279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:39.941776 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-44c62b3c-4234-4ded-892a-b044a9d50279 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:39.943012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-44c62b3c-4234-4ded-892a-b044a9d50279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:40.063239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 132/250] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:40.064021 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e01a5a-569d-421e-9cf2-8ef48de4fdfd None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:40.065020 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c62b3c-4234-4ded-892a-b044a9d50279 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:40.112381 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:40.126550 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f287470-0082-47cd-8bf8-0769cd6a5fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:40.135702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f287470-0082-47cd-8bf8-0769cd6a5fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:40.135702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f287470-0082-47cd-8bf8-0769cd6a5fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:40.135702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f287470-0082-47cd-8bf8-0769cd6a5fb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:40.135702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f287470-0082-47cd-8bf8-0769cd6a5fb1 None admin] RBAC: Adding query filter params (role.id=24179486bd6a4eb985a83d47ca5c2b4f, scope.project.id=21519c13797a4d5b8858f4eed4d738f8, user.id=c6d48aa625764ab9b2d848981827f091) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:40.135702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f287470-0082-47cd-8bf8-0769cd6a5fb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:40.135702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f287470-0082-47cd-8bf8-0769cd6a5fb1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:40.137404 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 119/251] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:24:39 2022] GET /identity/v3/role_assignments?user.id=c6d48aa625764ab9b2d848981827f091&scope.project.id=21519c13797a4d5b8858f4eed4d738f8&role.id=24179486bd6a4eb985a83d47ca5c2b4f => generated 259 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:40.139725 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f287470-0082-47cd-8bf8-0769cd6a5fb1 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:40.168364 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:40.295400 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c62b3c-4234-4ded-892a-b044a9d50279 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:40.304013 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f287470-0082-47cd-8bf8-0769cd6a5fb1 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:40.373799 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc37c3b-6160-4388-b4a0-a96b7f3a1d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:40.374822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc37c3b-6160-4388-b4a0-a96b7f3a1d54 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:40.375114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc37c3b-6160-4388-b4a0-a96b7f3a1d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:40.375389 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc37c3b-6160-4388-b4a0-a96b7f3a1d54 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:40.398162 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3c87a-97c8-494a-903f-85bbbddc7eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:40.398867 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3c87a-97c8-494a-903f-85bbbddc7eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:40.399157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3c87a-97c8-494a-903f-85bbbddc7eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:40.399432 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3c87a-97c8-494a-903f-85bbbddc7eb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:40.400354 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3c87a-97c8-494a-903f-85bbbddc7eb8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:40.400706 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3c87a-97c8-494a-903f-85bbbddc7eb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:40.402318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3c87a-97c8-494a-903f-85bbbddc7eb8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:40.409493 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc37c3b-6160-4388-b4a0-a96b7f3a1d54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:40.425913 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 133/252] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:40.432862 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3c87a-97c8-494a-903f-85bbbddc7eb8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:40.443714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-1cc37c3b-6160-4388-b4a0-a96b7f3a1d54 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:24:40.443714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:40.443714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:40.443714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:40.443714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:40.443714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:40.443714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:40.443714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:40.443714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:40.443714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:40.443714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:40.443714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:40.443714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:40.443714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:40.443714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:40.443714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:40.445535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:40.445535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:40.445535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:40.445535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:40.445535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:40.445535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:40.445535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:40.445535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:40.445535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:40.445535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:40.445535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:40.445535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:40.445535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:40.445535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:40.445535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:40.445535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:40.448320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:40.448320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:40.448320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:40.448320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:40.448320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:40.448320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:40.448320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:40.448320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:40.448320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:40.448320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:40.448320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:24:40.448320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:24:40.457884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 120/253] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:24:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:40.460081 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc37c3b-6160-4388-b4a0-a96b7f3a1d54 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:40.647114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:40.724227 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3c87a-97c8-494a-903f-85bbbddc7eb8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:40.743749 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:40.813571 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692f1c1-d08a-43cd-9f50-59926780a6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:40.814836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692f1c1-d08a-43cd-9f50-59926780a6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:40.815352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692f1c1-d08a-43cd-9f50-59926780a6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:40.815886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692f1c1-d08a-43cd-9f50-59926780a6b0 None admin] PATH_INFO: `/v3/users/d1243e4757d844928dd2cd699318de60` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:40.820516 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f692f1c1-d08a-43cd-9f50-59926780a6b0 None admin] RBAC: Authorizing `identity:get_user(user_id=d1243e4757d844928dd2cd699318de60)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:40.822406 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f692f1c1-d08a-43cd-9f50-59926780a6b0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:40.824584 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 134/254] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:40 2022] GET /identity/v3/users/d1243e4757d844928dd2cd699318de60 => generated 297 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:40.831201 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692f1c1-d08a-43cd-9f50-59926780a6b0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:40.863429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc37c3b-6160-4388-b4a0-a96b7f3a1d54 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:40.956681 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac256525-fd51-4b8a-8205-bcdaa9f35b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:40.958785 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac256525-fd51-4b8a-8205-bcdaa9f35b0f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:40.959559 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac256525-fd51-4b8a-8205-bcdaa9f35b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:40.961447 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac256525-fd51-4b8a-8205-bcdaa9f35b0f None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:40.962598 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac256525-fd51-4b8a-8205-bcdaa9f35b0f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:40.963895 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac256525-fd51-4b8a-8205-bcdaa9f35b0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:40.966660 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac256525-fd51-4b8a-8205-bcdaa9f35b0f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:40.985894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 121/255] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:24:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 527 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:40.992699 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac256525-fd51-4b8a-8205-bcdaa9f35b0f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:41.067296 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692f1c1-d08a-43cd-9f50-59926780a6b0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:41.142293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87447777-b2b4-4811-ab31-eb5660726b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:41.143569 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87447777-b2b4-4811-ab31-eb5660726b95 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:41.144862 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87447777-b2b4-4811-ab31-eb5660726b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:41.145521 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87447777-b2b4-4811-ab31-eb5660726b95 None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:41.148428 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87447777-b2b4-4811-ab31-eb5660726b95 None admin] RBAC: Authorizing `identity:get_project(project_id=21519c13797a4d5b8858f4eed4d738f8)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:41.151004 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87447777-b2b4-4811-ab31-eb5660726b95 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:41.154507 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 135/256] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:40 2022] GET /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8 => generated 302 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:24:41.164940 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87447777-b2b4-4811-ab31-eb5660726b95 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:41.261773 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac256525-fd51-4b8a-8205-bcdaa9f35b0f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:41.288538 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:41.289564 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:41.369477 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7c3b9-4706-44db-9686-b98158fb56a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:41.371153 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7c3b9-4706-44db-9686-b98158fb56a8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:41.371694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7c3b9-4706-44db-9686-b98158fb56a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:41.372206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7c3b9-4706-44db-9686-b98158fb56a8 None admin] PATH_INFO: `/v3/users/84438f0bf7944e0298f797fed3dd2622` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:41.374670 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7c3b9-4706-44db-9686-b98158fb56a8 None admin] RBAC: Authorizing `identity:get_user(user_id=84438f0bf7944e0298f797fed3dd2622)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:41.377580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7c3b9-4706-44db-9686-b98158fb56a8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:41.380939 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 122/257] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:40 2022] GET /identity/v3/users/84438f0bf7944e0298f797fed3dd2622 => generated 283 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:41.388724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7c3b9-4706-44db-9686-b98158fb56a8 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:41.441796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87447777-b2b4-4811-ab31-eb5660726b95 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:41.544869 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f8ce1-1842-4350-8ce1-f36180310879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:41.545705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f8ce1-1842-4350-8ce1-f36180310879 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:41.545995 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f8ce1-1842-4350-8ce1-f36180310879 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:41.546272 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f8ce1-1842-4350-8ce1-f36180310879 None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8/users/d1243e4757d844928dd2cd699318de60/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:41.548438 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f8ce1-1842-4350-8ce1-f36180310879 None admin] RBAC: Authorizing `identity:create_grant(project_id=21519c13797a4d5b8858f4eed4d738f8, user_id=d1243e4757d844928dd2cd699318de60, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:41.550113 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f8ce1-1842-4350-8ce1-f36180310879 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:41.658970 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 136/258] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:24:41 2022] PUT /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8/users/d1243e4757d844928dd2cd699318de60/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:24:41.711225 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7c3b9-4706-44db-9686-b98158fb56a8 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:41.747282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119c1b4-b0f9-4b39-a9f7-a95fddf2697d None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:41.747606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119c1b4-b0f9-4b39-a9f7-a95fddf2697d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:41.747885 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119c1b4-b0f9-4b39-a9f7-a95fddf2697d None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:41.749374 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 137/259] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:24:41.757676 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269c6bc-233d-4d63-b130-6e6ef9cf0770 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:41.757983 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269c6bc-233d-4d63-b130-6e6ef9cf0770 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:41.758237 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269c6bc-233d-4d63-b130-6e6ef9cf0770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:41.771060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b40828-1527-419f-8f3f-130e241be526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:41.771777 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b40828-1527-419f-8f3f-130e241be526 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:41.772066 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b40828-1527-419f-8f3f-130e241be526 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:41.772347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b40828-1527-419f-8f3f-130e241be526 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:41.792945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8269c6bc-233d-4d63-b130-6e6ef9cf0770 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:41.794212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8269c6bc-233d-4d63-b130-6e6ef9cf0770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:41.872137 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:41.894304 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b40828-1527-419f-8f3f-130e241be526 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:41.899966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b40828-1527-419f-8f3f-130e241be526 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:42.024964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 123/260] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 08:24:42.036806 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b40828-1527-419f-8f3f-130e241be526 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:42.041525 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 138/261] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:24:42.083925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269c6bc-233d-4d63-b130-6e6ef9cf0770 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:42.176176 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:42.193245 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:42.247692 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b40828-1527-419f-8f3f-130e241be526 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:42.325836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269c6bc-233d-4d63-b130-6e6ef9cf0770 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:42.362030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcfc94b-e341-4eeb-8b59-1d12da79a9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:42.363059 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfc94b-e341-4eeb-8b59-1d12da79a9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:42.363207 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfc94b-e341-4eeb-8b59-1d12da79a9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:42.363541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfc94b-e341-4eeb-8b59-1d12da79a9d3 None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:42.364869 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcfc94b-e341-4eeb-8b59-1d12da79a9d3 None admin] RBAC: Authorizing `identity:get_project(project_id=21519c13797a4d5b8858f4eed4d738f8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:42.366339 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcfc94b-e341-4eeb-8b59-1d12da79a9d3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:42.368816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 124/262] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:42 2022] GET /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8 => generated 302 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:42.376387 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcfc94b-e341-4eeb-8b59-1d12da79a9d3 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:42.418222 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2cbe96-474e-4616-9879-88ea6e305df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:42.419357 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cbe96-474e-4616-9879-88ea6e305df3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:42.419826 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cbe96-474e-4616-9879-88ea6e305df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:42.420278 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2cbe96-474e-4616-9879-88ea6e305df3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:42.448201 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2cbe96-474e-4616-9879-88ea6e305df3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:42.465543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-9e2cbe96-474e-4616-9879-88ea6e305df3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:24:42.465543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:42.465543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:42.465543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:42.465543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:42.465543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:42.465543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:42.465543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:42.465543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:42.465543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:42.465543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:42.465543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:42.465543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:42.465543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:42.465543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:42.465543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:42.466640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:42.466640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:42.466640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:42.466640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:42.466640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:42.466640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:42.466640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:42.466640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:42.466640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:42.466640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:42.466640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:42.466640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:42.466640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:42.466640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:42.466640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:42.466640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:42.467605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:42.467605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:42.467605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:42.467605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:42.467605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:42.467605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:42.467605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:42.467605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:42.467605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:42.467605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:42.467605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:24:42.467605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:42.470046 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 139/263] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:24:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:24:42.476393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2cbe96-474e-4616-9879-88ea6e305df3 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:42.660893 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcfc94b-e341-4eeb-8b59-1d12da79a9d3 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:42.795898 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688df41-1025-4851-8e21-0d30e311777c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:42.797394 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688df41-1025-4851-8e21-0d30e311777c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:42.798120 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688df41-1025-4851-8e21-0d30e311777c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:42.800933 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688df41-1025-4851-8e21-0d30e311777c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:42.802933 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688df41-1025-4851-8e21-0d30e311777c None admin] RBAC: Adding query filter params (role.id=d2ba35e60f974010a5e7dd5c4bebecf8, scope.project.id=21519c13797a4d5b8858f4eed4d738f8, user.id=84438f0bf7944e0298f797fed3dd2622) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:42.803747 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:42.807657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688df41-1025-4851-8e21-0d30e311777c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:42.811155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688df41-1025-4851-8e21-0d30e311777c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:42.830341 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 125/264] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:24:42 2022] GET /identity/v3/role_assignments?user.id=84438f0bf7944e0298f797fed3dd2622&scope.project.id=21519c13797a4d5b8858f4eed4d738f8&role.id=d2ba35e60f974010a5e7dd5c4bebecf8 => generated 608 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:24:42.835700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751287fc-ca49-4157-9cc0-0a8976c73c25 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:42.835700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751287fc-ca49-4157-9cc0-0a8976c73c25 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:42.835700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751287fc-ca49-4157-9cc0-0a8976c73c25 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:42.835700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 126/265] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:42.835700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e2eaa-b32d-4581-9d0f-fa52e955052b None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:42.835700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e2eaa-b32d-4581-9d0f-fa52e955052b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:42.836596 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e2eaa-b32d-4581-9d0f-fa52e955052b None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:42.836889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 127/266] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:42.843038 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fe247-06fc-42d2-91d2-9d9436697fb5 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:42.843546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fe247-06fc-42d2-91d2-9d9436697fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:42.844030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fe247-06fc-42d2-91d2-9d9436697fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:42.862572 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-5d7fe247-06fc-42d2-91d2-9d9436697fb5 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:42.864960 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5d7fe247-06fc-42d2-91d2-9d9436697fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:42.867904 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2cbe96-474e-4616-9879-88ea6e305df3 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:42.869658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:42.945422 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 128/267] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:42.951797 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695931f-1ade-43e6-9064-443064cbeeaf None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:42.956178 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695931f-1ade-43e6-9064-443064cbeeaf None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:42.956663 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695931f-1ade-43e6-9064-443064cbeeaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:42.982002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454de7f9-978c-4a95-a2bf-a7ea89ccd07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:42.983161 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454de7f9-978c-4a95-a2bf-a7ea89ccd07a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:42.983908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454de7f9-978c-4a95-a2bf-a7ea89ccd07a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:42.984616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454de7f9-978c-4a95-a2bf-a7ea89ccd07a None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:42.986267 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454de7f9-978c-4a95-a2bf-a7ea89ccd07a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:42.987043 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454de7f9-978c-4a95-a2bf-a7ea89ccd07a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:42.989906 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454de7f9-978c-4a95-a2bf-a7ea89ccd07a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:42.999704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-5695931f-1ade-43e6-9064-443064cbeeaf None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:42.999704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5695931f-1ade-43e6-9064-443064cbeeaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:43.027631 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 140/268] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 552 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:24:43.029564 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454de7f9-978c-4a95-a2bf-a7ea89ccd07a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:43.076016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 129/269] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:43.077128 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695931f-1ade-43e6-9064-443064cbeeaf None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:43.159186 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:43.196284 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695931f-1ade-43e6-9064-443064cbeeaf None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:43.252918 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454de7f9-978c-4a95-a2bf-a7ea89ccd07a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:43.279114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4930ac1-178a-4f46-854b-af4c9bd194af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:43.280092 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4930ac1-178a-4f46-854b-af4c9bd194af None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:43.280377 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4930ac1-178a-4f46-854b-af4c9bd194af None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:43.280846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4930ac1-178a-4f46-854b-af4c9bd194af None admin] PATH_INFO: `/v3/users/84438f0bf7944e0298f797fed3dd2622` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:43.282553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4930ac1-178a-4f46-854b-af4c9bd194af None admin] RBAC: Authorizing `identity:get_user(user_id=84438f0bf7944e0298f797fed3dd2622)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:43.284354 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4930ac1-178a-4f46-854b-af4c9bd194af None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:43.286323 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 130/270] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:43 2022] GET /identity/v3/users/84438f0bf7944e0298f797fed3dd2622 => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:43.287298 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4930ac1-178a-4f46-854b-af4c9bd194af None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:43.324453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f1573-9712-490f-b181-d7d35e6049b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:43.326182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f1573-9712-490f-b181-d7d35e6049b5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:43.326813 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f1573-9712-490f-b181-d7d35e6049b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:43.327604 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f1573-9712-490f-b181-d7d35e6049b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:43.330364 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f1573-9712-490f-b181-d7d35e6049b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:43.333588 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f1573-9712-490f-b181-d7d35e6049b5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:43.337267 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 141/271] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:24:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:43.339821 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f1573-9712-490f-b181-d7d35e6049b5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:43.391739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:43.417909 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4930ac1-178a-4f46-854b-af4c9bd194af None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:43.460808 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ed118-4f01-495b-a38d-6e1f81643623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:43.461987 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ed118-4f01-495b-a38d-6e1f81643623 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:43.462569 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ed118-4f01-495b-a38d-6e1f81643623 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:43.463629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ed118-4f01-495b-a38d-6e1f81643623 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:43.472478 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:43.478138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ed118-4f01-495b-a38d-6e1f81643623 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:43.487443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-031ed118-4f01-495b-a38d-6e1f81643623 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:43.487443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:43.487443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:43.487443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:43.487443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:43.487443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:43.487443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:43.487443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:43.487443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:43.487443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:43.487443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:43.487443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:43.487443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:43.487443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:43.487443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:43.487443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:43.488448 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:43.488448 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:43.488448 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:43.488448 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:43.488448 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:43.488448 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:43.488448 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:43.488448 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:43.488448 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:43.488448 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:43.488448 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:43.488448 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:43.488448 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:43.488448 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:43.488448 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:43.488448 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:43.489255 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:43.489255 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:43.489255 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:43.489255 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:43.489255 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:43.489255 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:43.489255 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:43.489255 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:43.489255 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:43.489255 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:43.489255 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:43.489255 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:24:43.493447 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 131/272] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:43.495333 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ed118-4f01-495b-a38d-6e1f81643623 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:43.546430 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f1573-9712-490f-b181-d7d35e6049b5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:43.614750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a96e7f-8ff0-40c1-a592-770f844922ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:43.616824 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a96e7f-8ff0-40c1-a592-770f844922ef None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:43.617322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a96e7f-8ff0-40c1-a592-770f844922ef None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:43.617789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a96e7f-8ff0-40c1-a592-770f844922ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:43.619494 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ed118-4f01-495b-a38d-6e1f81643623 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:43.669138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c6d19-dafa-4401-9667-748e58352a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:43.669938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c6d19-dafa-4401-9667-748e58352a16 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:43.670429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c6d19-dafa-4401-9667-748e58352a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:43.670923 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c6d19-dafa-4401-9667-748e58352a16 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:43.671898 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c6d19-dafa-4401-9667-748e58352a16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:43.673593 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c6d19-dafa-4401-9667-748e58352a16 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:43.756179 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 132/273] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 18 08:24:43 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:24:43.757594 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c6d19-dafa-4401-9667-748e58352a16 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:43.763755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a96e7f-8ff0-40c1-a592-770f844922ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:43.763755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a96e7f-8ff0-40c1-a592-770f844922ef None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:43.809878 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:43.848282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:43.903108 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c6d19-dafa-4401-9667-748e58352a16 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:43.945491 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceedd3e-96e0-4a06-862b-96d01b5fd5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:43.946451 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceedd3e-96e0-4a06-862b-96d01b5fd5ff None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:43.947628 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceedd3e-96e0-4a06-862b-96d01b5fd5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:43.948197 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceedd3e-96e0-4a06-862b-96d01b5fd5ff None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:43.949243 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceedd3e-96e0-4a06-862b-96d01b5fd5ff None admin] RBAC: Adding query filter params (name=member) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:43.949798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceedd3e-96e0-4a06-862b-96d01b5fd5ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:43.951411 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceedd3e-96e0-4a06-862b-96d01b5fd5ff None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:43.963290 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 133/274] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:43.982798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceedd3e-96e0-4a06-862b-96d01b5fd5ff None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:44.081837 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 142/275] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 741 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:24:44.106305 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a96e7f-8ff0-40c1-a592-770f844922ef None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:44.239993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceedd3e-96e0-4a06-862b-96d01b5fd5ff None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:44.279327 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:44.316662 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f99e8c-38e3-429a-b480-a2b6d0c1f484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:44.318439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f99e8c-38e3-429a-b480-a2b6d0c1f484 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:44.319563 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f99e8c-38e3-429a-b480-a2b6d0c1f484 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:44.320302 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f99e8c-38e3-429a-b480-a2b6d0c1f484 None admin] PATH_INFO: `/v3/users/1b5fc95d6b2c49d4aaadc1d147dd6ac6` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:44.337239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a96e7f-8ff0-40c1-a592-770f844922ef None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:44.349490 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f99e8c-38e3-429a-b480-a2b6d0c1f484 None admin] RBAC: Authorizing `identity:get_user(user_id=1b5fc95d6b2c49d4aaadc1d147dd6ac6)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:44.352157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f99e8c-38e3-429a-b480-a2b6d0c1f484 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:44.355899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 134/276] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:43 2022] GET /identity/v3/users/1b5fc95d6b2c49d4aaadc1d147dd6ac6 => generated 293 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:24:44.358244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc44eb-0f81-4bd1-a22e-45c3583fc213 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:44.358725 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc44eb-0f81-4bd1-a22e-45c3583fc213 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:44.359373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc44eb-0f81-4bd1-a22e-45c3583fc213 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:44.362415 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 135/277] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:44.372582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937526b0-5b7b-4dc3-8d18-679a58e0cfeb None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:44.373037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937526b0-5b7b-4dc3-8d18-679a58e0cfeb None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:44.373402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937526b0-5b7b-4dc3-8d18-679a58e0cfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:44.393604 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:44.402946 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-937526b0-5b7b-4dc3-8d18-679a58e0cfeb None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:44.404878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-937526b0-5b7b-4dc3-8d18-679a58e0cfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:44.418899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c6316-19d8-45f9-98d7-04a12428fb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:44.420396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c6316-19d8-45f9-98d7-04a12428fb8f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:44.420877 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c6316-19d8-45f9-98d7-04a12428fb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:44.421362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c6316-19d8-45f9-98d7-04a12428fb8f None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:44.426910 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c6316-19d8-45f9-98d7-04a12428fb8f None admin] RBAC: Authorizing `identity:get_project(project_id=21519c13797a4d5b8858f4eed4d738f8)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:44.429631 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c6316-19d8-45f9-98d7-04a12428fb8f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:44.433080 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 143/278] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:44 2022] GET /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8 => generated 302 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:24:44.434357 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c6316-19d8-45f9-98d7-04a12428fb8f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:44.569540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 136/279] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:44.571229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937526b0-5b7b-4dc3-8d18-679a58e0cfeb None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:44.673755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c6316-19d8-45f9-98d7-04a12428fb8f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:44.720134 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b2b31-7249-468f-81df-4a59bfb4be2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:44.721214 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b2b31-7249-468f-81df-4a59bfb4be2c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:44.721733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b2b31-7249-468f-81df-4a59bfb4be2c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:44.722220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b2b31-7249-468f-81df-4a59bfb4be2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:44.723302 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b2b31-7249-468f-81df-4a59bfb4be2c None admin] RBAC: Adding query filter params (role.id=587af95739954600bbf2411974601f74, user.id=1b5fc95d6b2c49d4aaadc1d147dd6ac6, scope.system=all) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:44.723869 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b2b31-7249-468f-81df-4a59bfb4be2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:44.725561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b2b31-7249-468f-81df-4a59bfb4be2c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:44.733433 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 144/280] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Jul 18 08:24:44 2022] GET /identity/v3/role_assignments?user.id=1b5fc95d6b2c49d4aaadc1d147dd6ac6&scope.system=all&role.id=587af95739954600bbf2411974601f74 => generated 226 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:44.734622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b2b31-7249-468f-81df-4a59bfb4be2c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:44.788160 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937526b0-5b7b-4dc3-8d18-679a58e0cfeb None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:44.793654 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:44.885819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:44.913122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19e4aa-50e9-4410-98cb-ff739efa19a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:44.914250 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19e4aa-50e9-4410-98cb-ff739efa19a4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:44.914820 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19e4aa-50e9-4410-98cb-ff739efa19a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:44.915301 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19e4aa-50e9-4410-98cb-ff739efa19a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:44.916458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19e4aa-50e9-4410-98cb-ff739efa19a4 None admin] RBAC: Adding query filter params (role.id=467d49b8a0d3435aaba58464cc8ef85c, scope.project.id=21519c13797a4d5b8858f4eed4d738f8, user.id=84438f0bf7944e0298f797fed3dd2622) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:44.917033 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19e4aa-50e9-4410-98cb-ff739efa19a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:44.918740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19e4aa-50e9-4410-98cb-ff739efa19a4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:44.939104 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 137/281] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:24:44 2022] GET /identity/v3/role_assignments?user.id=84438f0bf7944e0298f797fed3dd2622&scope.project.id=21519c13797a4d5b8858f4eed4d738f8&role.id=467d49b8a0d3435aaba58464cc8ef85c => generated 608 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:44.992019 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b2b31-7249-468f-81df-4a59bfb4be2c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:45.069416 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364209fd-16c8-45b7-8394-7771ebda47de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:45.070756 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364209fd-16c8-45b7-8394-7771ebda47de None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:45.071270 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364209fd-16c8-45b7-8394-7771ebda47de None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:45.072429 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364209fd-16c8-45b7-8394-7771ebda47de None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:45.102253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364209fd-16c8-45b7-8394-7771ebda47de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:45.123394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-364209fd-16c8-45b7-8394-7771ebda47de None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 08:24:45.123394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:45.123394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:45.123394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:45.123394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:45.123394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:45.123394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:45.123394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:45.123394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:45.123394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:45.123394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:45.123394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:45.123394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:45.123394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:45.123394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:45.123394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:45.124380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:45.124380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:45.124380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:45.124380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:45.124380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:45.124380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:45.124380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:45.124380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:45.124380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:45.124380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:45.124380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:45.124380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:45.124380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:45.124380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:45.124380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:45.124380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:45.125178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:45.125178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:45.125178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:45.125178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:45.125178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:45.125178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:45.125178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:45.125178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:45.125178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:45.125178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:45.125178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 08:24:45.125178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:45.125797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 145/282] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:45.133552 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19e4aa-50e9-4410-98cb-ff739efa19a4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:45.288364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19e4aa-50e9-4410-98cb-ff739efa19a4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:45.373754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ba323-42b7-456b-ade6-c47a14c31b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:45.375282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ba323-42b7-456b-ade6-c47a14c31b7d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:45.376098 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ba323-42b7-456b-ade6-c47a14c31b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:45.376890 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ba323-42b7-456b-ade6-c47a14c31b7d None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:45.378781 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ba323-42b7-456b-ade6-c47a14c31b7d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:45.379704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ba323-42b7-456b-ade6-c47a14c31b7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:45.382703 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ba323-42b7-456b-ade6-c47a14c31b7d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:45.406630 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 138/283] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:45.413614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364209fd-16c8-45b7-8394-7771ebda47de None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:45.610262 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364209fd-16c8-45b7-8394-7771ebda47de None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:45.638272 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:45.675903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33829eec-29c6-4066-b684-6d9b03dfa609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:45.677744 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33829eec-29c6-4066-b684-6d9b03dfa609 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:45.678278 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33829eec-29c6-4066-b684-6d9b03dfa609 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:45.678789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33829eec-29c6-4066-b684-6d9b03dfa609 None admin] PATH_INFO: `/v3/users/c6d48aa625764ab9b2d848981827f091` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:45.680739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33829eec-29c6-4066-b684-6d9b03dfa609 None admin] RBAC: Authorizing `identity:get_user(user_id=c6d48aa625764ab9b2d848981827f091)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:45.683656 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33829eec-29c6-4066-b684-6d9b03dfa609 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:45.686253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 146/284] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:45 2022] GET /identity/v3/users/c6d48aa625764ab9b2d848981827f091 => generated 297 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:45.691649 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ba323-42b7-456b-ade6-c47a14c31b7d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:45.731406 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:45.740057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6c349-d97e-40a2-9229-ae7803aec560 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:45.740448 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6c349-d97e-40a2-9229-ae7803aec560 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:45.742281 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6c349-d97e-40a2-9229-ae7803aec560 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:45.742281 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 147/285] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:24:45.748515 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473926f4-a986-4e74-b6c2-3a7e20df4fd9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:45.748772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473926f4-a986-4e74-b6c2-3a7e20df4fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:45.749154 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473926f4-a986-4e74-b6c2-3a7e20df4fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:45.777601 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-473926f4-a986-4e74-b6c2-3a7e20df4fd9 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:45.778885 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-473926f4-a986-4e74-b6c2-3a7e20df4fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:45.963685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ba323-42b7-456b-ade6-c47a14c31b7d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:45.993831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 148/286] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:24:46.017585 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473926f4-a986-4e74-b6c2-3a7e20df4fd9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:46.034366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f073be3-a5a8-476f-934e-6580a970e4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:46.035257 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f073be3-a5a8-476f-934e-6580a970e4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:46.035535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f073be3-a5a8-476f-934e-6580a970e4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:46.038880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f073be3-a5a8-476f-934e-6580a970e4a1 None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:46.041813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f073be3-a5a8-476f-934e-6580a970e4a1 None admin] RBAC: Authorizing `identity:get_project(project_id=21519c13797a4d5b8858f4eed4d738f8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:46.047552 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f073be3-a5a8-476f-934e-6580a970e4a1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:46.049581 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 139/287] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:45 2022] GET /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8 => generated 302 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:24:46.055059 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f073be3-a5a8-476f-934e-6580a970e4a1 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:46.195043 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473926f4-a986-4e74-b6c2-3a7e20df4fd9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:46.216617 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:46.235263 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:46.241759 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f073be3-a5a8-476f-934e-6580a970e4a1 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:46.273622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226e2b0-7d12-4994-9bef-3eaea56837da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:46.274468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226e2b0-7d12-4994-9bef-3eaea56837da None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:46.274867 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226e2b0-7d12-4994-9bef-3eaea56837da None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:46.275734 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226e2b0-7d12-4994-9bef-3eaea56837da None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:46.295916 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d226e2b0-7d12-4994-9bef-3eaea56837da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:46.307074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d701040-bc19-402d-93d2-3df88587e4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:46.308027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d701040-bc19-402d-93d2-3df88587e4b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:46.308312 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d701040-bc19-402d-93d2-3df88587e4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:46.308608 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d701040-bc19-402d-93d2-3df88587e4b4 None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8/users/c6d48aa625764ab9b2d848981827f091/roles/24179486bd6a4eb985a83d47ca5c2b4f` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:46.310769 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d701040-bc19-402d-93d2-3df88587e4b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=21519c13797a4d5b8858f4eed4d738f8, user_id=c6d48aa625764ab9b2d848981827f091, role_id=24179486bd6a4eb985a83d47ca5c2b4f)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:46.312263 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d701040-bc19-402d-93d2-3df88587e4b4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:46.314624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-d226e2b0-7d12-4994-9bef-3eaea56837da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:46.314624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:46.314624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:46.314624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:46.314624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:46.314624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:46.314624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:46.314624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:46.314624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:46.314624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:46.314624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:46.314624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:46.314624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:46.314624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:46.314624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:46.314624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:46.316474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:46.316474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:46.316474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:46.316474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:46.316474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:46.316474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:46.316474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:46.316474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:46.316474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:46.316474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:46.316474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:46.316474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:46.316474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:46.316474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:46.316474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:46.316474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:46.317997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:46.317997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:46.317997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:46.317997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:46.317997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:46.317997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:46.317997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:46.317997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:46.317997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:46.317997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:46.317997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:46.317997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:46.317997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 149/288] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:24:46.328870 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226e2b0-7d12-4994-9bef-3eaea56837da None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:46.403436 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 140/289] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:24:46 2022] PUT /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8/users/c6d48aa625764ab9b2d848981827f091/roles/24179486bd6a4eb985a83d47ca5c2b4f => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:24:46.528168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226e2b0-7d12-4994-9bef-3eaea56837da None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:46.581600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27939697-34c6-4745-a832-bc83f1549dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:46.582727 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27939697-34c6-4745-a832-bc83f1549dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:46.583233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27939697-34c6-4745-a832-bc83f1549dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:46.583750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27939697-34c6-4745-a832-bc83f1549dc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:46.584825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27939697-34c6-4745-a832-bc83f1549dc8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:46.585373 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27939697-34c6-4745-a832-bc83f1549dc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:46.587052 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27939697-34c6-4745-a832-bc83f1549dc8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:46.599205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 150/290] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:24:46.606262 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d701040-bc19-402d-93d2-3df88587e4b4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:46.733790 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d701040-bc19-402d-93d2-3df88587e4b4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:46.777049 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7f64c-0c71-4c08-b9d5-a353e1edb091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:46.778498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7f64c-0c71-4c08-b9d5-a353e1edb091 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:46.779011 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7f64c-0c71-4c08-b9d5-a353e1edb091 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:46.779485 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7f64c-0c71-4c08-b9d5-a353e1edb091 None admin] PATH_INFO: `/v3/users/d1243e4757d844928dd2cd699318de60` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:46.781167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d7f64c-0c71-4c08-b9d5-a353e1edb091 None admin] RBAC: Authorizing `identity:get_user(user_id=d1243e4757d844928dd2cd699318de60)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:46.783001 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d7f64c-0c71-4c08-b9d5-a353e1edb091 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:46.785100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 141/291] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:46 2022] GET /identity/v3/users/d1243e4757d844928dd2cd699318de60 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:46.791287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27939697-34c6-4745-a832-bc83f1549dc8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:46.853855 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:46.944320 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27939697-34c6-4745-a832-bc83f1549dc8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:46.953714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f406f53-c1cd-4811-b12b-16d6ad237aeb None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:46.954184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f406f53-c1cd-4811-b12b-16d6ad237aeb None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:46.954660 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f406f53-c1cd-4811-b12b-16d6ad237aeb None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:46.957240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 142/292] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:46.967033 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252e463-0241-4726-bbc3-93015a19bb63 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:46.967537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252e463-0241-4726-bbc3-93015a19bb63 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:46.968082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252e463-0241-4726-bbc3-93015a19bb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:46.987221 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2368a-a68b-44a6-8cd3-a1866d6aed1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:46.988523 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2368a-a68b-44a6-8cd3-a1866d6aed1c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:46.988810 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2368a-a68b-44a6-8cd3-a1866d6aed1c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:46.989085 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2368a-a68b-44a6-8cd3-a1866d6aed1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:46.998574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-7252e463-0241-4726-bbc3-93015a19bb63 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:47.001086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7252e463-0241-4726-bbc3-93015a19bb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:47.046638 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:47.143007 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2368a-a68b-44a6-8cd3-a1866d6aed1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:47.144727 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2368a-a68b-44a6-8cd3-a1866d6aed1c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:47.197350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:47.199603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 143/293] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:24:47.274599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252e463-0241-4726-bbc3-93015a19bb63 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:47.294339 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 151/294] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 08:24:47.304987 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2368a-a68b-44a6-8cd3-a1866d6aed1c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:47.444559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2368a-a68b-44a6-8cd3-a1866d6aed1c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:47.487641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252e463-0241-4726-bbc3-93015a19bb63 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:47.541917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d0d6a-76c6-45e0-a0ac-2982f3fd3b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:47.542804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d0d6a-76c6-45e0-a0ac-2982f3fd3b2a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:47.543081 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d0d6a-76c6-45e0-a0ac-2982f3fd3b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:47.543366 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d0d6a-76c6-45e0-a0ac-2982f3fd3b2a None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:47.544777 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d0d6a-76c6-45e0-a0ac-2982f3fd3b2a None admin] RBAC: Authorizing `identity:get_project(project_id=21519c13797a4d5b8858f4eed4d738f8)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:47.546277 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d0d6a-76c6-45e0-a0ac-2982f3fd3b2a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:47.546441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048873c-25c7-4f46-a2ac-53630a961c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:47.547127 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048873c-25c7-4f46-a2ac-53630a961c0b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:47.547416 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048873c-25c7-4f46-a2ac-53630a961c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:47.547727 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048873c-25c7-4f46-a2ac-53630a961c0b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:47.548106 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 152/295] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:47 2022] GET /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:47.549399 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3841c-4fe7-45da-8e15-334c8ff738f4 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:47.550107 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3841c-4fe7-45da-8e15-334c8ff738f4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:47.550393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3841c-4fe7-45da-8e15-334c8ff738f4 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:47.552293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 153/296] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:47.558946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ec67e-e712-46be-90ac-22ef3d86f382 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:47.559166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ec67e-e712-46be-90ac-22ef3d86f382 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:47.559541 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ec67e-e712-46be-90ac-22ef3d86f382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:47.563932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048873c-25c7-4f46-a2ac-53630a961c0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:47.565162 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:47.574708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-8048873c-25c7-4f46-a2ac-53630a961c0b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 08:24:47.574708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:47.574708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:47.574708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:47.574708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:47.574708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:47.574708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:47.574708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:47.574708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:47.574708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:47.574708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:47.574708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:47.574708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:47.574708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:47.574708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:47.574708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:47.585674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:47.585674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:47.585674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:47.585674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:47.585674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:47.585674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:47.585674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:47.585674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:47.585674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:47.585674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:47.585674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:47.585674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:47.585674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:47.585674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:47.585674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:47.585674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:47.588498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:47.588498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:47.588498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:47.588498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:47.588498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:47.588498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:47.588498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:47.588498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:47.588498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:47.588498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:47.588498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 08:24:47.588498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:24:47.588498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 144/297] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:24:47.588498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048873c-25c7-4f46-a2ac-53630a961c0b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:47.591039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-717ec67e-e712-46be-90ac-22ef3d86f382 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:47.591039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-717ec67e-e712-46be-90ac-22ef3d86f382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:47.604984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:47.692935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 154/298] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:47.693992 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ec67e-e712-46be-90ac-22ef3d86f382 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:47.739001 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048873c-25c7-4f46-a2ac-53630a961c0b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:47.789672 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5ab61-0fc1-4264-82e6-2fa4b2a48b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:47.791274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5ab61-0fc1-4264-82e6-2fa4b2a48b74 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:47.791817 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5ab61-0fc1-4264-82e6-2fa4b2a48b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:47.792616 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5ab61-0fc1-4264-82e6-2fa4b2a48b74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:47.794048 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5ab61-0fc1-4264-82e6-2fa4b2a48b74 None admin] RBAC: Adding query filter params (role.id=587af95739954600bbf2411974601f74, scope.project.id=21519c13797a4d5b8858f4eed4d738f8, user.id=d1243e4757d844928dd2cd699318de60) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:47.794646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5ab61-0fc1-4264-82e6-2fa4b2a48b74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:47.796386 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5ab61-0fc1-4264-82e6-2fa4b2a48b74 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:47.805318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 145/299] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:24:47 2022] GET /identity/v3/role_assignments?user.id=d1243e4757d844928dd2cd699318de60&scope.project.id=21519c13797a4d5b8858f4eed4d738f8&role.id=587af95739954600bbf2411974601f74 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:47.807720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5ab61-0fc1-4264-82e6-2fa4b2a48b74 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:47.877748 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ec67e-e712-46be-90ac-22ef3d86f382 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:47.949539 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:47.971966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5ab61-0fc1-4264-82e6-2fa4b2a48b74 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:48.011620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:48.016713 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3261c57-0a35-4a4a-a7b0-09f535a84a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:48.017148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3261c57-0a35-4a4a-a7b0-09f535a84a21 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:48.017435 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3261c57-0a35-4a4a-a7b0-09f535a84a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:48.017711 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3261c57-0a35-4a4a-a7b0-09f535a84a21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:48.018979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3261c57-0a35-4a4a-a7b0-09f535a84a21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:48.020818 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3261c57-0a35-4a4a-a7b0-09f535a84a21 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:48.024020 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 146/300] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:24:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:48.024132 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901d7af-a454-4aab-afa2-652d5361bc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:48.025502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901d7af-a454-4aab-afa2-652d5361bc53 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:48.025980 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901d7af-a454-4aab-afa2-652d5361bc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:48.026440 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901d7af-a454-4aab-afa2-652d5361bc53 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:48.027476 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901d7af-a454-4aab-afa2-652d5361bc53 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:48.028004 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901d7af-a454-4aab-afa2-652d5361bc53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:48.029185 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3261c57-0a35-4a4a-a7b0-09f535a84a21 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:48.030052 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901d7af-a454-4aab-afa2-652d5361bc53 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:48.041177 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 155/301] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:48.048316 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901d7af-a454-4aab-afa2-652d5361bc53 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:48.155688 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3261c57-0a35-4a4a-a7b0-09f535a84a21 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:48.181125 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901d7af-a454-4aab-afa2-652d5361bc53 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:48.195371 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439035b2-f87c-4294-8ed1-a1dc3ba169cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:48.196110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439035b2-f87c-4294-8ed1-a1dc3ba169cb None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:48.196404 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439035b2-f87c-4294-8ed1-a1dc3ba169cb None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:48.196734 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439035b2-f87c-4294-8ed1-a1dc3ba169cb None admin] PATH_INFO: `/v3/groups` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:48.197601 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439035b2-f87c-4294-8ed1-a1dc3ba169cb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:48.199088 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439035b2-f87c-4294-8ed1-a1dc3ba169cb None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:48.227521 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c0901-5abc-469d-96f2-dfa536580126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:48.229368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c0901-5abc-469d-96f2-dfa536580126 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:48.229717 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c0901-5abc-469d-96f2-dfa536580126 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:48.230003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c0901-5abc-469d-96f2-dfa536580126 None admin] PATH_INFO: `/v3/users/bc75dc182793411da6de0f9b128337ae` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:48.252874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c0901-5abc-469d-96f2-dfa536580126 None admin] RBAC: Authorizing `identity:get_user(user_id=bc75dc182793411da6de0f9b128337ae)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:48.254899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c0901-5abc-469d-96f2-dfa536580126 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:48.257120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 156/302] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:48 2022] GET /identity/v3/users/bc75dc182793411da6de0f9b128337ae => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:48.258527 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82ac34-18dc-424a-a4af-46e30e5b4e25 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:48.258938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82ac34-18dc-424a-a4af-46e30e5b4e25 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:48.259372 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82ac34-18dc-424a-a4af-46e30e5b4e25 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:48.261215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 157/303] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:48.266085 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82ac34-18dc-424a-a4af-46e30e5b4e25 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:48.293193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 147/304] 10.222.0.57 () {60 vars in 1134 bytes} [Mon Jul 18 08:24:48 2022] POST /identity/v3/groups => generated 229 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:24:48.296295 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a798ef7-533c-4adb-9642-0fe4f27e6bb6 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:48.296763 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a798ef7-533c-4adb-9642-0fe4f27e6bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:48.297205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a798ef7-533c-4adb-9642-0fe4f27e6bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:48.329086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-3a798ef7-533c-4adb-9642-0fe4f27e6bb6 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:48.331048 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3a798ef7-533c-4adb-9642-0fe4f27e6bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:48.334384 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:48.376030 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:48.463172 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 148/305] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:48.484318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82ac34-18dc-424a-a4af-46e30e5b4e25 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:48.501064 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a798ef7-533c-4adb-9642-0fe4f27e6bb6 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:48.548694 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5084b1c-3fb4-484a-911d-97621ab35c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:48.549383 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5084b1c-3fb4-484a-911d-97621ab35c3f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:48.549668 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5084b1c-3fb4-484a-911d-97621ab35c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:48.550276 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5084b1c-3fb4-484a-911d-97621ab35c3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:48.551250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5084b1c-3fb4-484a-911d-97621ab35c3f None admin] RBAC: Adding query filter params (role.id=24179486bd6a4eb985a83d47ca5c2b4f, user.id=bc75dc182793411da6de0f9b128337ae, scope.system=all) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:48.552171 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5084b1c-3fb4-484a-911d-97621ab35c3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:48.555872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5084b1c-3fb4-484a-911d-97621ab35c3f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:48.561937 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 158/306] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Jul 18 08:24:48 2022] GET /identity/v3/role_assignments?user.id=bc75dc182793411da6de0f9b128337ae&scope.system=all&role.id=24179486bd6a4eb985a83d47ca5c2b4f => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:48.658699 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a798ef7-533c-4adb-9642-0fe4f27e6bb6 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:48.705257 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d710b-91b1-4214-81b4-a96a2903c797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:48.706496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d710b-91b1-4214-81b4-a96a2903c797 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:48.706961 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d710b-91b1-4214-81b4-a96a2903c797 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:48.707393 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d710b-91b1-4214-81b4-a96a2903c797 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:48.717423 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:48.724091 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d710b-91b1-4214-81b4-a96a2903c797 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:48.733977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-962d710b-91b1-4214-81b4-a96a2903c797 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:48.733977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:48.733977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:48.733977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:48.733977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:48.733977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:48.733977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:48.733977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:48.733977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:48.733977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:48.733977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:48.733977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:48.733977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:48.733977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:48.733977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:48.733977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:48.735110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:48.735110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:48.735110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:48.735110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:48.735110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:48.735110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:48.735110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:48.735110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:48.735110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:48.735110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:48.735110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:48.735110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:48.735110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:48.735110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:48.735110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:48.735110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:48.736095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:48.736095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:48.736095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:48.736095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:48.736095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:48.736095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:48.736095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:48.736095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:48.736095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:48.736095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:48.736095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:48.736095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:24:48.738632 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 149/307] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:24:48.748462 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5084b1c-3fb4-484a-911d-97621ab35c3f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:48.941918 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5084b1c-3fb4-484a-911d-97621ab35c3f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:48.969336 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:48.988828 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7d61d-359c-447d-a3c4-864a4cd4e3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:48.989908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7d61d-359c-447d-a3c4-864a4cd4e3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:48.990398 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7d61d-359c-447d-a3c4-864a4cd4e3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:48.990901 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7d61d-359c-447d-a3c4-864a4cd4e3c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:48.992163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7d61d-359c-447d-a3c4-864a4cd4e3c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:48.992684 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7d61d-359c-447d-a3c4-864a4cd4e3c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:48.996090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7d61d-359c-447d-a3c4-864a4cd4e3c4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:49.006471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 159/308] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:49.016041 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d710b-91b1-4214-81b4-a96a2903c797 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:49.210236 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d710b-91b1-4214-81b4-a96a2903c797 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:49.277165 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd4b79-a304-4778-8f79-4bb4e32b003d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:49.278597 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd4b79-a304-4778-8f79-4bb4e32b003d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:49.279077 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd4b79-a304-4778-8f79-4bb4e32b003d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:49.279548 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd4b79-a304-4778-8f79-4bb4e32b003d None admin] PATH_INFO: `/v3/users/1b5fc95d6b2c49d4aaadc1d147dd6ac6` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:49.281835 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd4b79-a304-4778-8f79-4bb4e32b003d None admin] RBAC: Authorizing `identity:get_user(user_id=1b5fc95d6b2c49d4aaadc1d147dd6ac6)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:49.284585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd4b79-a304-4778-8f79-4bb4e32b003d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:49.287794 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 150/309] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:49 2022] GET /identity/v3/users/1b5fc95d6b2c49d4aaadc1d147dd6ac6 => generated 293 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:49.294471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7d61d-359c-447d-a3c4-864a4cd4e3c4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:49.476169 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7d61d-359c-447d-a3c4-864a4cd4e3c4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:49.538439 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cc00d-9a49-42b7-8b71-a6dde78cfbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:49.539365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75f363-a021-4753-b333-4685a8d9b7ba None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:49.539964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cc00d-9a49-42b7-8b71-a6dde78cfbe0 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:49.540079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75f363-a021-4753-b333-4685a8d9b7ba None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:49.544455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75f363-a021-4753-b333-4685a8d9b7ba None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:49.544661 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cc00d-9a49-42b7-8b71-a6dde78cfbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:49.544661 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cc00d-9a49-42b7-8b71-a6dde78cfbe0 None admin] PATH_INFO: `/v3/system/users/1b5fc95d6b2c49d4aaadc1d147dd6ac6/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:49.544918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 151/310] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:49.547742 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cc00d-9a49-42b7-8b71-a6dde78cfbe0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1b5fc95d6b2c49d4aaadc1d147dd6ac6, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:49.548448 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cc00d-9a49-42b7-8b71-a6dde78cfbe0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:49.559729 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c9e2e-5e6d-4674-8d90-e8a9148c4f48 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:49.559729 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c9e2e-5e6d-4674-8d90-e8a9148c4f48 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:49.559729 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c9e2e-5e6d-4674-8d90-e8a9148c4f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:49.589642 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-222c9e2e-5e6d-4674-8d90-e8a9148c4f48 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:49.591960 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-222c9e2e-5e6d-4674-8d90-e8a9148c4f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:49.650502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 160/311] 10.222.0.57 () {58 vars in 1334 bytes} [Mon Jul 18 08:24:49 2022] PUT /identity/v3/system/users/1b5fc95d6b2c49d4aaadc1d147dd6ac6/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:24:49.727515 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:49.744032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 152/312] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:49.786493 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cc00d-9a49-42b7-8b71-a6dde78cfbe0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:49.876671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:49.995949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cc00d-9a49-42b7-8b71-a6dde78cfbe0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:50.057508 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f8d28-1d14-4091-8251-4f0e5c662ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:50.058727 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f8d28-1d14-4091-8251-4f0e5c662ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:50.059215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f8d28-1d14-4091-8251-4f0e5c662ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:50.059741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f8d28-1d14-4091-8251-4f0e5c662ba1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:50.072969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f8d28-1d14-4091-8251-4f0e5c662ba1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:50.081355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-1a2f8d28-1d14-4091-8251-4f0e5c662ba1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 08:24:50.081355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:50.081355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:50.081355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:50.081355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:50.081355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:50.081355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:50.081355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:50.081355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:50.081355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:50.081355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:50.081355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:50.081355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:50.081355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:50.081355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:50.081355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:50.082290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:50.082290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:50.082290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:50.082290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:50.082290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:50.082290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:50.082290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:50.082290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:50.082290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:50.082290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:50.082290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:50.082290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:50.082290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:50.082290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:50.082290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:50.082290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:50.083092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:50.083092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:50.083092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:50.083092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:50.083092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:50.083092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:50.083092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:50.083092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:50.083092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:50.083092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:50.083092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 08:24:50.083092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:50.085925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 161/313] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:50.095596 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c9e2e-5e6d-4674-8d90-e8a9148c4f48 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:50.258388 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c9e2e-5e6d-4674-8d90-e8a9148c4f48 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:50.297705 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9467b2-86b3-427c-9418-6d6f84b77ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:50.298745 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9467b2-86b3-427c-9418-6d6f84b77ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:50.299235 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9467b2-86b3-427c-9418-6d6f84b77ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:50.299765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9467b2-86b3-427c-9418-6d6f84b77ce2 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:50.300731 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9467b2-86b3-427c-9418-6d6f84b77ce2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:50.301240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9467b2-86b3-427c-9418-6d6f84b77ce2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:50.302814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9467b2-86b3-427c-9418-6d6f84b77ce2 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:50.312828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 153/314] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:50.322295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f8d28-1d14-4091-8251-4f0e5c662ba1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:50.511001 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f8d28-1d14-4091-8251-4f0e5c662ba1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:50.513922 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:50.573495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693a307-e7ef-49a7-ade2-2549a3e483b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:50.574445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693a307-e7ef-49a7-ade2-2549a3e483b1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:50.574733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693a307-e7ef-49a7-ade2-2549a3e483b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:50.575010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693a307-e7ef-49a7-ade2-2549a3e483b1 None admin] PATH_INFO: `/v3/users/c6d48aa625764ab9b2d848981827f091` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:50.576397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693a307-e7ef-49a7-ade2-2549a3e483b1 None admin] RBAC: Authorizing `identity:get_user(user_id=c6d48aa625764ab9b2d848981827f091)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:50.577898 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693a307-e7ef-49a7-ade2-2549a3e483b1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:50.580025 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 162/315] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:50 2022] GET /identity/v3/users/c6d48aa625764ab9b2d848981827f091 => generated 297 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:24:50.585834 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9467b2-86b3-427c-9418-6d6f84b77ce2 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:50.689989 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:50.773261 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9467b2-86b3-427c-9418-6d6f84b77ce2 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:50.848004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cba2c-c161-449c-a4db-54af4215d06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:50.849097 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cba2c-c161-449c-a4db-54af4215d06c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:50.849619 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cba2c-c161-449c-a4db-54af4215d06c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:50.850158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cba2c-c161-449c-a4db-54af4215d06c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:50.931630 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cba2c-c161-449c-a4db-54af4215d06c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:50.933596 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cba2c-c161-449c-a4db-54af4215d06c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:51.007959 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:51.057019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 154/316] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:24:51.063504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693a307-e7ef-49a7-ade2-2549a3e483b1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:51.087914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88494d99-a51f-4106-8c14-e709bd0846b7 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:51.088355 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88494d99-a51f-4106-8c14-e709bd0846b7 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:51.088818 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88494d99-a51f-4106-8c14-e709bd0846b7 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:51.091456 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 155/317] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:51.100072 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c612925-6942-4c73-ad25-8069d2ec4a91 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:51.100682 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c612925-6942-4c73-ad25-8069d2ec4a91 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:51.101063 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c612925-6942-4c73-ad25-8069d2ec4a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:51.128169 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-1c612925-6942-4c73-ad25-8069d2ec4a91 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:51.130209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1c612925-6942-4c73-ad25-8069d2ec4a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:51.208537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693a307-e7ef-49a7-ade2-2549a3e483b1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:51.236042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 156/318] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:51.237654 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d0b65-4f0c-4a00-9f99-3df08fe30bfe None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:51.237990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d0b65-4f0c-4a00-9f99-3df08fe30bfe None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:51.238232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d0b65-4f0c-4a00-9f99-3df08fe30bfe None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:51.239753 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 157/319] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:51.247420 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2d000-ad27-4773-b64d-fb66f4b0c16e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:51.247728 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2d000-ad27-4773-b64d-fb66f4b0c16e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:51.248072 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2d000-ad27-4773-b64d-fb66f4b0c16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:51.272540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-cfa2d000-ad27-4773-b64d-fb66f4b0c16e None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:51.273900 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-cfa2d000-ad27-4773-b64d-fb66f4b0c16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:51.277680 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023f0f3-0e54-4920-9410-a7f1b9021420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:51.278773 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023f0f3-0e54-4920-9410-a7f1b9021420 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:51.279183 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023f0f3-0e54-4920-9410-a7f1b9021420 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:51.279594 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023f0f3-0e54-4920-9410-a7f1b9021420 None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:51.281184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023f0f3-0e54-4920-9410-a7f1b9021420 None admin] RBAC: Authorizing `identity:get_project(project_id=21519c13797a4d5b8858f4eed4d738f8)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:51.282830 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023f0f3-0e54-4920-9410-a7f1b9021420 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:51.285057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 163/320] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:51 2022] GET /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:51.291763 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023f0f3-0e54-4920-9410-a7f1b9021420 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:51.351498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:51.408187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 158/321] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:51.410181 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2d000-ad27-4773-b64d-fb66f4b0c16e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:51.425142 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023f0f3-0e54-4920-9410-a7f1b9021420 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:51.453871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:51.477514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332365c0-9632-40ff-9eaa-f7e7f0d9b25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:51.479438 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332365c0-9632-40ff-9eaa-f7e7f0d9b25d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:51.479975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332365c0-9632-40ff-9eaa-f7e7f0d9b25d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:51.480471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332365c0-9632-40ff-9eaa-f7e7f0d9b25d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:51.499332 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332365c0-9632-40ff-9eaa-f7e7f0d9b25d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:51.514415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-332365c0-9632-40ff-9eaa-f7e7f0d9b25d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 08:24:51.514415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:51.514415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:51.514415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:51.514415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:51.514415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:51.514415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:51.514415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:51.514415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:51.514415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:51.514415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:51.514415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:51.514415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:51.514415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:51.514415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:51.514415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:51.515397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:51.515397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:51.515397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:51.515397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:51.515397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:51.515397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:51.515397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:51.515397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:51.515397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:51.515397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:51.515397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:51.515397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:51.515397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:51.515397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:51.515397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:51.515397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:51.516238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:51.516238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:51.516238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:51.516238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:51.516238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:51.516238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:51.516238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:51.516238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:51.516238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:51.516238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:51.516238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 08:24:51.516238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:51.516854 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 164/322] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:51.517869 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332365c0-9632-40ff-9eaa-f7e7f0d9b25d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:51.582130 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2d000-ad27-4773-b64d-fb66f4b0c16e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:51.642495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d572b-976e-4272-b67d-1062a9e10f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:51.643493 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d572b-976e-4272-b67d-1062a9e10f31 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:51.644007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d572b-976e-4272-b67d-1062a9e10f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:51.645015 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d572b-976e-4272-b67d-1062a9e10f31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:51.645976 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d572b-976e-4272-b67d-1062a9e10f31 None admin] RBAC: Adding query filter params (role.id=24179486bd6a4eb985a83d47ca5c2b4f, scope.project.id=21519c13797a4d5b8858f4eed4d738f8, user.id=c6d48aa625764ab9b2d848981827f091) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:51.646478 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d572b-976e-4272-b67d-1062a9e10f31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:51.649090 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d572b-976e-4272-b67d-1062a9e10f31 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:51.661035 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 159/323] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:24:51 2022] GET /identity/v3/role_assignments?user.id=c6d48aa625764ab9b2d848981827f091&scope.project.id=21519c13797a4d5b8858f4eed4d738f8&role.id=24179486bd6a4eb985a83d47ca5c2b4f => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:51.662443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d572b-976e-4272-b67d-1062a9e10f31 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:51.707741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:51.715557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332365c0-9632-40ff-9eaa-f7e7f0d9b25d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:51.796414 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb7633-fb81-4db1-8094-1fd455135c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:51.798463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb7633-fb81-4db1-8094-1fd455135c86 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:51.799183 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb7633-fb81-4db1-8094-1fd455135c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:51.800403 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb7633-fb81-4db1-8094-1fd455135c86 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:51.803238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb7633-fb81-4db1-8094-1fd455135c86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:51.807768 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb7633-fb81-4db1-8094-1fd455135c86 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:51.812074 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 165/324] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:24:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:51.822643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb7633-fb81-4db1-8094-1fd455135c86 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:51.851953 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:51.882885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d572b-976e-4272-b67d-1062a9e10f31 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:51.960491 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50582dcc-0ff9-44c9-9253-efe1a90ddd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:51.961634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50582dcc-0ff9-44c9-9253-efe1a90ddd8f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:51.962703 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50582dcc-0ff9-44c9-9253-efe1a90ddd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:51.963167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50582dcc-0ff9-44c9-9253-efe1a90ddd8f None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:51.964553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50582dcc-0ff9-44c9-9253-efe1a90ddd8f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:51.965088 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50582dcc-0ff9-44c9-9253-efe1a90ddd8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:51.967858 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50582dcc-0ff9-44c9-9253-efe1a90ddd8f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:51.982005 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 160/325] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:51.990652 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50582dcc-0ff9-44c9-9253-efe1a90ddd8f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:52.056424 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb7633-fb81-4db1-8094-1fd455135c86 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:52.126838 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2ff99-3a3f-4e62-b1fa-555da2a5d350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:52.129119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2ff99-3a3f-4e62-b1fa-555da2a5d350 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:52.129595 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2ff99-3a3f-4e62-b1fa-555da2a5d350 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:52.130069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2ff99-3a3f-4e62-b1fa-555da2a5d350 None admin] PATH_INFO: `/v3/groups` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:52.131343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2ff99-3a3f-4e62-b1fa-555da2a5d350 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:52.134629 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2ff99-3a3f-4e62-b1fa-555da2a5d350 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:52.151346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50582dcc-0ff9-44c9-9253-efe1a90ddd8f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:52.194418 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36ec08-be7c-46db-b6a5-d6cc31996790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:52.195978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36ec08-be7c-46db-b6a5-d6cc31996790 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:52.196984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36ec08-be7c-46db-b6a5-d6cc31996790 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:52.198049 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36ec08-be7c-46db-b6a5-d6cc31996790 None admin] PATH_INFO: `/v3/users/bc75dc182793411da6de0f9b128337ae` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:52.200441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c36ec08-be7c-46db-b6a5-d6cc31996790 None admin] RBAC: Authorizing `identity:get_user(user_id=bc75dc182793411da6de0f9b128337ae)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:52.203229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c36ec08-be7c-46db-b6a5-d6cc31996790 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:52.206259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 161/326] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:51 2022] GET /identity/v3/users/bc75dc182793411da6de0f9b128337ae => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:52.214195 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36ec08-be7c-46db-b6a5-d6cc31996790 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:52.223993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 166/327] 10.222.0.57 () {60 vars in 1134 bytes} [Mon Jul 18 08:24:51 2022] POST /identity/v3/groups => generated 226 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:24:52.225431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecf9be-ec1c-4710-ae30-16a20e4ee92f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:52.225730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecf9be-ec1c-4710-ae30-16a20e4ee92f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:52.226418 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecf9be-ec1c-4710-ae30-16a20e4ee92f None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:52.227723 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 167/328] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:52.233596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a25e89-a872-4a1c-8e5d-6a2e7902b6f8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:52.234100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a25e89-a872-4a1c-8e5d-6a2e7902b6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:52.234581 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a25e89-a872-4a1c-8e5d-6a2e7902b6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:52.238249 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:52.256355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-99a25e89-a872-4a1c-8e5d-6a2e7902b6f8 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:52.257741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-99a25e89-a872-4a1c-8e5d-6a2e7902b6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:52.286444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:52.359933 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 168/329] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:24:52.371695 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36ec08-be7c-46db-b6a5-d6cc31996790 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:52.384764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a25e89-a872-4a1c-8e5d-6a2e7902b6f8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:52.418907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cb958-b254-4798-b20b-7b3e6b4c4456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:52.419867 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cb958-b254-4798-b20b-7b3e6b4c4456 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:52.420153 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cb958-b254-4798-b20b-7b3e6b4c4456 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:52.420416 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cb958-b254-4798-b20b-7b3e6b4c4456 None admin] PATH_INFO: `/v3/system/users/bc75dc182793411da6de0f9b128337ae/roles/24179486bd6a4eb985a83d47ca5c2b4f` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:52.422043 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8cb958-b254-4798-b20b-7b3e6b4c4456 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bc75dc182793411da6de0f9b128337ae, role_id=24179486bd6a4eb985a83d47ca5c2b4f)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:52.423471 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8cb958-b254-4798-b20b-7b3e6b4c4456 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:52.500467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 162/330] 10.222.0.57 () {58 vars in 1334 bytes} [Mon Jul 18 08:24:52 2022] PUT /identity/v3/system/users/bc75dc182793411da6de0f9b128337ae/roles/24179486bd6a4eb985a83d47ca5c2b4f => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:24:52.533163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a25e89-a872-4a1c-8e5d-6a2e7902b6f8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:52.604208 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ff0bf-b4d8-4518-97cf-335650d1d0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:52.605749 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ff0bf-b4d8-4518-97cf-335650d1d0ca None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:52.606276 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ff0bf-b4d8-4518-97cf-335650d1d0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:52.606756 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ff0bf-b4d8-4518-97cf-335650d1d0ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:52.629674 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ff0bf-b4d8-4518-97cf-335650d1d0ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:52.645117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-996ff0bf-b4d8-4518-97cf-335650d1d0ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:52.645117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:52.645117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:52.645117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:52.645117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:52.645117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:52.645117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:52.645117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:52.645117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:52.645117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:52.645117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:52.645117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:52.645117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:52.645117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:52.645117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:52.645117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:52.646167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:52.646167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:52.646167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:52.646167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:52.646167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:52.646167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:52.646167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:52.646167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:52.646167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:52.646167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:52.646167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:52.646167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:52.646167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:52.646167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:52.646167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:52.646167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:52.646993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:52.646993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:52.646993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:52.646993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:52.646993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:52.646993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:52.646993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:52.646993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:52.646993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:52.646993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:52.646993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:52.646993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:52.648185 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 169/331] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:52.657214 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cb958-b254-4798-b20b-7b3e6b4c4456 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:52.784153 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:52.831911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cb958-b254-4798-b20b-7b3e6b4c4456 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:52.898592 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf810ff-0fa1-455a-b252-54f0aa5fff2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:52.899299 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf810ff-0fa1-455a-b252-54f0aa5fff2c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:52.903746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf810ff-0fa1-455a-b252-54f0aa5fff2c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:52.903746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf810ff-0fa1-455a-b252-54f0aa5fff2c None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:52.903746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf810ff-0fa1-455a-b252-54f0aa5fff2c None admin] RBAC: Adding query filter params (name=member) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:52.903746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf810ff-0fa1-455a-b252-54f0aa5fff2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:52.903746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf810ff-0fa1-455a-b252-54f0aa5fff2c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:52.914173 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 163/332] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:52.921164 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ff0bf-b4d8-4518-97cf-335650d1d0ca None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:52.950316 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:53.071354 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ff0bf-b4d8-4518-97cf-335650d1d0ca None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:53.122760 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8e6ee-b361-4ec2-84de-3a8474e8de7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:53.123692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8e6ee-b361-4ec2-84de-3a8474e8de7c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:53.123996 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8e6ee-b361-4ec2-84de-3a8474e8de7c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:53.124281 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8e6ee-b361-4ec2-84de-3a8474e8de7c None admin] PATH_INFO: `/v3/users/1b5fc95d6b2c49d4aaadc1d147dd6ac6` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:53.125750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8e6ee-b361-4ec2-84de-3a8474e8de7c None admin] RBAC: Authorizing `identity:get_user(user_id=1b5fc95d6b2c49d4aaadc1d147dd6ac6)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:53.127333 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8e6ee-b361-4ec2-84de-3a8474e8de7c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:53.129214 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 170/333] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:52 2022] GET /identity/v3/users/1b5fc95d6b2c49d4aaadc1d147dd6ac6 => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:53.137246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf810ff-0fa1-455a-b252-54f0aa5fff2c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:53.338178 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf810ff-0fa1-455a-b252-54f0aa5fff2c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:53.401084 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97feab4f-dc59-46c2-b549-1371a646f50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:53.401830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97feab4f-dc59-46c2-b549-1371a646f50e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:53.402121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97feab4f-dc59-46c2-b549-1371a646f50e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:53.402765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97feab4f-dc59-46c2-b549-1371a646f50e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:53.403321 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97feab4f-dc59-46c2-b549-1371a646f50e None admin] RBAC: Adding query filter params (role.id=587af95739954600bbf2411974601f74, user.id=1b5fc95d6b2c49d4aaadc1d147dd6ac6, scope.system=all) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:53.403641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97feab4f-dc59-46c2-b549-1371a646f50e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:53.405158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97feab4f-dc59-46c2-b549-1371a646f50e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:53.410993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:53.420680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 164/334] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Jul 18 08:24:53 2022] GET /identity/v3/role_assignments?user.id=1b5fc95d6b2c49d4aaadc1d147dd6ac6&scope.system=all&role.id=587af95739954600bbf2411974601f74 => generated 510 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:55.354091 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052c720-bd4a-4ddb-a32c-daaee70e52e8 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:55.354634 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052c720-bd4a-4ddb-a32c-daaee70e52e8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:55.354634 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052c720-bd4a-4ddb-a32c-daaee70e52e8 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:55.356349 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 171/335] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:55.364927 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f98b93-f2cd-4753-8861-197a487795dc None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:55.365426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f98b93-f2cd-4753-8861-197a487795dc None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:55.365525 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f98b93-f2cd-4753-8861-197a487795dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:55.387834 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-26f98b93-f2cd-4753-8861-197a487795dc None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:55.389072 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-26f98b93-f2cd-4753-8861-197a487795dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:55.484656 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 165/336] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:55.548969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052c720-bd4a-4ddb-a32c-daaee70e52e8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:55.670079 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:55.694179 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052c720-bd4a-4ddb-a32c-daaee70e52e8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:55.743983 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f28949-091c-4028-9e81-5d4b3f754920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:55.744857 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f28949-091c-4028-9e81-5d4b3f754920 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:55.745146 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f28949-091c-4028-9e81-5d4b3f754920 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:55.745428 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f28949-091c-4028-9e81-5d4b3f754920 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:55.763366 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f28949-091c-4028-9e81-5d4b3f754920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:55.771724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-54f28949-091c-4028-9e81-5d4b3f754920 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:55.771724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:55.771724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:55.771724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:55.771724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:55.771724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:55.771724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:55.771724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:55.771724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:55.771724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:55.771724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:55.771724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:55.771724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:55.771724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:55.771724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:55.771724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:55.775466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:55.775466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:55.775466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:55.775466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:55.775466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:55.775466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:55.775466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:55.775466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:55.775466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:55.775466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:55.775466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:55.775466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:55.775466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:55.775466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:55.775466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:55.775466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:55.776315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:55.776315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:55.776315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:55.776315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:55.776315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:55.776315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:55.776315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:55.776315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:55.776315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:55.776315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:55.776315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:55.776315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:55.779713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 172/337] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:55.786014 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f98b93-f2cd-4753-8861-197a487795dc None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:55.927456 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f98b93-f2cd-4753-8861-197a487795dc None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:55.961292 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:55.972160 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4531149-6db3-4c61-b8b8-75a39276eedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:55.975540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4531149-6db3-4c61-b8b8-75a39276eedf None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:55.975540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4531149-6db3-4c61-b8b8-75a39276eedf None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:55.975540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4531149-6db3-4c61-b8b8-75a39276eedf None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:55.975540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4531149-6db3-4c61-b8b8-75a39276eedf None admin] RBAC: Adding query filter params (name=member) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:55.975540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4531149-6db3-4c61-b8b8-75a39276eedf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:55.975917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4531149-6db3-4c61-b8b8-75a39276eedf None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:55.988302 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 166/338] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:55.997695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f28949-091c-4028-9e81-5d4b3f754920 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:56.133004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b057c-d376-4d18-aa64-50da4e4bb02c None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:56.133334 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b057c-d376-4d18-aa64-50da4e4bb02c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:56.133581 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b057c-d376-4d18-aa64-50da4e4bb02c None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:56.135259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 167/339] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:56.141737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f28949-091c-4028-9e81-5d4b3f754920 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:56.146727 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf683d1-4755-4daa-b8ce-639781cb0ffa None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:56.147180 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf683d1-4755-4daa-b8ce-639781cb0ffa None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:56.147605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf683d1-4755-4daa-b8ce-639781cb0ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:56.195358 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb2ea4c-d7e2-48d0-a2e3-43931b8ff334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:56.198090 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-3bf683d1-4755-4daa-b8ce-639781cb0ffa None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:56.200093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3bf683d1-4755-4daa-b8ce-639781cb0ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:56.204567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2ea4c-d7e2-48d0-a2e3-43931b8ff334 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:56.205019 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2ea4c-d7e2-48d0-a2e3-43931b8ff334 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:56.205457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2ea4c-d7e2-48d0-a2e3-43931b8ff334 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:56.248544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:56.364796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 168/340] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:56.370885 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2ea4c-d7e2-48d0-a2e3-43931b8ff334 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:56.373022 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2ea4c-d7e2-48d0-a2e3-43931b8ff334 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:56.409180 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf683d1-4755-4daa-b8ce-639781cb0ffa None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:56.519448 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 173/341] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:24:56.521108 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f15cc-380a-446a-95f8-e5038ed421df None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:56.521561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f15cc-380a-446a-95f8-e5038ed421df None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:56.522006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f15cc-380a-446a-95f8-e5038ed421df None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:56.523761 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 174/342] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:56.532666 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e4897-7567-411d-a02e-ee4f2af9feb2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:56.533271 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e4897-7567-411d-a02e-ee4f2af9feb2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:56.533735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e4897-7567-411d-a02e-ee4f2af9feb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:56.568924 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-982e4897-7567-411d-a02e-ee4f2af9feb2 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:56.570475 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-982e4897-7567-411d-a02e-ee4f2af9feb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:56.632972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:56.640881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf683d1-4755-4daa-b8ce-639781cb0ffa None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:56.667836 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:56.692249 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426360e-6de6-48dc-8f77-a537c8dc01a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:56.693447 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426360e-6de6-48dc-8f77-a537c8dc01a4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:56.693957 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426360e-6de6-48dc-8f77-a537c8dc01a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:56.694383 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426360e-6de6-48dc-8f77-a537c8dc01a4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:56.697949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 175/343] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:56.700201 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e4897-7567-411d-a02e-ee4f2af9feb2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:56.708766 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6426360e-6de6-48dc-8f77-a537c8dc01a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:56.719870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-6426360e-6de6-48dc-8f77-a537c8dc01a4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 08:24:56.719870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:56.719870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:56.719870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:56.719870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:56.719870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:56.719870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:56.719870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:56.719870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:56.719870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:56.719870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:56.719870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:56.719870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:56.719870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:56.719870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:56.719870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:56.721032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:56.721032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:56.721032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:56.721032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:56.721032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:56.721032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:56.721032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:56.721032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:56.721032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:56.721032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:56.721032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:56.721032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:56.721032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:56.721032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:56.721032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:56.721032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:56.721982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:56.721982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:56.721982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:56.721982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:56.721982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:56.721982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:56.721982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:56.721982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:56.721982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:56.721982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:56.721982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 08:24:56.721982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:24:56.724841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 169/344] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:56.731720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426360e-6de6-48dc-8f77-a537c8dc01a4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:56.972059 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e4897-7567-411d-a02e-ee4f2af9feb2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:57.029367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426360e-6de6-48dc-8f77-a537c8dc01a4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:57.055097 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc3a89-b389-4d04-8e7e-0f5f78471f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:57.056852 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3a89-b389-4d04-8e7e-0f5f78471f04 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:57.057363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3a89-b389-4d04-8e7e-0f5f78471f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:57.057865 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3a89-b389-4d04-8e7e-0f5f78471f04 None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:57.060227 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc3a89-b389-4d04-8e7e-0f5f78471f04 None admin] RBAC: Authorizing `identity:get_project(project_id=791daf53d0364b258d9cfb153ffa3111)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:57.063114 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc3a89-b389-4d04-8e7e-0f5f78471f04 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:57.068381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 176/345] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:56 2022] GET /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111 => generated 298 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:57.070032 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc3a89-b389-4d04-8e7e-0f5f78471f04 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:57.070472 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d6195-988c-49f3-943e-4e68ceda5670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:57.071527 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d6195-988c-49f3-943e-4e68ceda5670 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:57.072085 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d6195-988c-49f3-943e-4e68ceda5670 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:57.072672 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d6195-988c-49f3-943e-4e68ceda5670 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:57.073675 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d6195-988c-49f3-943e-4e68ceda5670 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:57.074195 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d6195-988c-49f3-943e-4e68ceda5670 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:57.076184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d6195-988c-49f3-943e-4e68ceda5670 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:57.086716 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 170/346] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:24:57.089746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ef78b-e69b-4ab5-80da-912841ba16a4 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:57.090299 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ef78b-e69b-4ab5-80da-912841ba16a4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:57.090795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ef78b-e69b-4ab5-80da-912841ba16a4 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:57.097494 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 171/347] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:57.101120 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ef78b-e69b-4ab5-80da-912841ba16a4 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:57.165984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:57.188232 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:57.256834 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ef78b-e69b-4ab5-80da-912841ba16a4 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:57.303702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2632ce-481c-4457-88a3-0567780f650a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:57.304643 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2632ce-481c-4457-88a3-0567780f650a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:57.304951 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2632ce-481c-4457-88a3-0567780f650a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:57.307567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2632ce-481c-4457-88a3-0567780f650a None admin] PATH_INFO: `/v3/groups/640b54e919004efebb102e0e48da8ceb` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:57.315407 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2632ce-481c-4457-88a3-0567780f650a None admin] RBAC: Authorizing `identity:get_group(group_id=640b54e919004efebb102e0e48da8ceb)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:57.316951 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2632ce-481c-4457-88a3-0567780f650a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:57.318702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 172/348] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 18 08:24:57 2022] GET /identity/v3/groups/640b54e919004efebb102e0e48da8ceb => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:57.319565 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2632ce-481c-4457-88a3-0567780f650a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:57.337537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc3a89-b389-4d04-8e7e-0f5f78471f04 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:57.412913 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fce4c9-fe2c-4c90-9e9a-32245b9ef585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:57.413811 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fce4c9-fe2c-4c90-9e9a-32245b9ef585 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:57.413931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fce4c9-fe2c-4c90-9e9a-32245b9ef585 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:57.414304 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fce4c9-fe2c-4c90-9e9a-32245b9ef585 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:57.429200 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fce4c9-fe2c-4c90-9e9a-32245b9ef585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:57.440090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-99fce4c9-fe2c-4c90-9e9a-32245b9ef585 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:57.440090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:57.440090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:57.440090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:57.440090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:57.440090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:57.440090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:57.440090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:57.440090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:57.440090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:57.440090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:57.440090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:57.440090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:57.440090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:57.440090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:57.440090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:57.441874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:57.441874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:57.441874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:57.441874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:57.441874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:57.441874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:57.441874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:57.441874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:57.441874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:57.441874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:57.441874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:57.441874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:57.441874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:57.441874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:57.441874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:57.441874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:57.443313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:57.443313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:57.443313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:57.443313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:57.443313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:57.443313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:57.443313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:57.443313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:57.443313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:57.443313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:57.443313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:24:57.443313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:57.443313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 177/349] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:57.443313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d918aea-b2c9-4228-b097-e7d6dd39bedf None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:57.450637 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d918aea-b2c9-4228-b097-e7d6dd39bedf None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:57.450637 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d918aea-b2c9-4228-b097-e7d6dd39bedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:57.478927 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3d918aea-b2c9-4228-b097-e7d6dd39bedf None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:57.481698 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3d918aea-b2c9-4228-b097-e7d6dd39bedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:57.518016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2632ce-481c-4457-88a3-0567780f650a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:57.521136 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:57.569762 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec99e2-a0a3-4a0c-8e59-12b014b966ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:57.570762 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec99e2-a0a3-4a0c-8e59-12b014b966ba None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:57.571647 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec99e2-a0a3-4a0c-8e59-12b014b966ba None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:57.574571 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec99e2-a0a3-4a0c-8e59-12b014b966ba None admin] PATH_INFO: `/v3/users/bc75dc182793411da6de0f9b128337ae` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:57.576392 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec99e2-a0a3-4a0c-8e59-12b014b966ba None admin] RBAC: Authorizing `identity:get_user(user_id=bc75dc182793411da6de0f9b128337ae)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:57.578274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec99e2-a0a3-4a0c-8e59-12b014b966ba None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:57.580680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 173/350] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:24:57 2022] GET /identity/v3/users/bc75dc182793411da6de0f9b128337ae => generated 293 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:57.584937 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec99e2-a0a3-4a0c-8e59-12b014b966ba None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:57.632540 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 178/351] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:57.634853 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d918aea-b2c9-4228-b097-e7d6dd39bedf None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:57.700270 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:57.731367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec99e2-a0a3-4a0c-8e59-12b014b966ba None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:57.785113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36042fa1-12c1-418e-9edc-a6c372cd0442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:57.785804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36042fa1-12c1-418e-9edc-a6c372cd0442 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:57.786089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36042fa1-12c1-418e-9edc-a6c372cd0442 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:57.786357 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36042fa1-12c1-418e-9edc-a6c372cd0442 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:57.787247 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36042fa1-12c1-418e-9edc-a6c372cd0442 None admin] RBAC: Adding query filter params (group.id=640b54e919004efebb102e0e48da8ceb, role.id=587af95739954600bbf2411974601f74, scope.project.id=791daf53d0364b258d9cfb153ffa3111) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:57.787567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36042fa1-12c1-418e-9edc-a6c372cd0442 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:57.789196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36042fa1-12c1-418e-9edc-a6c372cd0442 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:57.796499 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 174/352] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 18 08:24:57 2022] GET /identity/v3/role_assignments?group.id=640b54e919004efebb102e0e48da8ceb&scope.project.id=791daf53d0364b258d9cfb153ffa3111&role.id=587af95739954600bbf2411974601f74 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:57.797536 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36042fa1-12c1-418e-9edc-a6c372cd0442 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:57.851731 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d918aea-b2c9-4228-b097-e7d6dd39bedf None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:57.882555 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:57.926237 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed4c25-e594-4c91-8b55-47598dc13732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:57.927369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed4c25-e594-4c91-8b55-47598dc13732 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:57.927912 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed4c25-e594-4c91-8b55-47598dc13732 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:57.928403 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed4c25-e594-4c91-8b55-47598dc13732 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:57.929930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed4c25-e594-4c91-8b55-47598dc13732 None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:57.930507 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed4c25-e594-4c91-8b55-47598dc13732 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:57.933122 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed4c25-e594-4c91-8b55-47598dc13732 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:57.946163 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36042fa1-12c1-418e-9edc-a6c372cd0442 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:57.952266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 179/353] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:24:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:57.954029 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed4c25-e594-4c91-8b55-47598dc13732 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:57.990076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532679c8-186e-4a23-93a4-1bed9205c818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:57.990770 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532679c8-186e-4a23-93a4-1bed9205c818 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:57.991439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532679c8-186e-4a23-93a4-1bed9205c818 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:57.991795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532679c8-186e-4a23-93a4-1bed9205c818 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:57.992647 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532679c8-186e-4a23-93a4-1bed9205c818 None admin] RBAC: Adding query filter params (role.id=24179486bd6a4eb985a83d47ca5c2b4f, user.id=bc75dc182793411da6de0f9b128337ae, scope.system=all) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:57.993112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532679c8-186e-4a23-93a4-1bed9205c818 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:57.994760 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532679c8-186e-4a23-93a4-1bed9205c818 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:58.008497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 175/354] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Jul 18 08:24:57 2022] GET /identity/v3/role_assignments?user.id=bc75dc182793411da6de0f9b128337ae&scope.system=all&role.id=24179486bd6a4eb985a83d47ca5c2b4f => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:58.009422 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532679c8-186e-4a23-93a4-1bed9205c818 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:58.132128 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:58.141605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532679c8-186e-4a23-93a4-1bed9205c818 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:58.142622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:58.155630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed4c25-e594-4c91-8b55-47598dc13732 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:58.196708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884aab9-f359-40af-84aa-4a42cdd80f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:58.197404 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884aab9-f359-40af-84aa-4a42cdd80f26 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:58.197686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884aab9-f359-40af-84aa-4a42cdd80f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:58.198216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884aab9-f359-40af-84aa-4a42cdd80f26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:58.230681 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f48246-538e-4494-ad2b-bf44d634706a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:58.232202 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f48246-538e-4494-ad2b-bf44d634706a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:58.232826 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f48246-538e-4494-ad2b-bf44d634706a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:58.233435 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f48246-538e-4494-ad2b-bf44d634706a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:58.259218 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f48246-538e-4494-ad2b-bf44d634706a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:58.269385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-e5f48246-538e-4494-ad2b-bf44d634706a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:24:58.269385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:58.269385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:58.269385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:58.269385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:58.269385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:58.269385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:58.269385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:58.269385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:58.269385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:58.269385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:58.269385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:58.269385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:58.269385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:58.269385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:58.269385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:58.271655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:58.271655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:58.271655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:58.271655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:58.271655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:58.271655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:58.271655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:58.271655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:58.271655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:58.271655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:58.271655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:58.271655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:58.271655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:58.271655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:58.271655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:58.271655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:58.273502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:58.273502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:58.273502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:58.273502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:58.273502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:58.273502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:58.273502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:58.273502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:58.273502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:58.273502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:58.273502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:24:58.273502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:58.273502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 180/355] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:24:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:58.273502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719ed7e-6ae5-4985-8e6a-41ab5765e0d5 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:58.276252 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719ed7e-6ae5-4985-8e6a-41ab5765e0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:58.276252 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719ed7e-6ae5-4985-8e6a-41ab5765e0d5 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:58.276252 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 181/356] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:58.277643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719ed7e-6ae5-4985-8e6a-41ab5765e0d5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:58.338075 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884aab9-f359-40af-84aa-4a42cdd80f26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:58.342817 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884aab9-f359-40af-84aa-4a42cdd80f26 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:58.462738 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719ed7e-6ae5-4985-8e6a-41ab5765e0d5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:58.530388 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:58.531117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5cb66c-51c6-4574-8a52-0fb1ee81b416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:58.531855 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cb66c-51c6-4574-8a52-0fb1ee81b416 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:58.532983 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cb66c-51c6-4574-8a52-0fb1ee81b416 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:58.532983 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cb66c-51c6-4574-8a52-0fb1ee81b416 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:58.533497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5cb66c-51c6-4574-8a52-0fb1ee81b416 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:58.533907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5cb66c-51c6-4574-8a52-0fb1ee81b416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:58.535453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5cb66c-51c6-4574-8a52-0fb1ee81b416 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:58.549045 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 182/357] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:24:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:58.550537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 176/358] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:24:58.550814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c55439-7112-48e5-9486-6afa20ba6f21 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:58.550954 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c55439-7112-48e5-9486-6afa20ba6f21 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:58.551262 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c55439-7112-48e5-9486-6afa20ba6f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:58.554197 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8dd09-884c-4c40-b73e-1954c14e8891 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:58.555076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8dd09-884c-4c40-b73e-1954c14e8891 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:58.555882 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8dd09-884c-4c40-b73e-1954c14e8891 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:58.560527 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 177/359] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:24:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:24:58.562384 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8dd09-884c-4c40-b73e-1954c14e8891 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:58.574669 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-45c55439-7112-48e5-9486-6afa20ba6f21 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:58.578093 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-45c55439-7112-48e5-9486-6afa20ba6f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:58.599169 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:58.704341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 183/360] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:58.705220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c55439-7112-48e5-9486-6afa20ba6f21 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:58.748441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8dd09-884c-4c40-b73e-1954c14e8891 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:58.803075 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cabe7-9d72-4ccb-8cc7-f7a4bbff77f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:58.804461 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cabe7-9d72-4ccb-8cc7-f7a4bbff77f1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:58.804741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cabe7-9d72-4ccb-8cc7-f7a4bbff77f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:58.805012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cabe7-9d72-4ccb-8cc7-f7a4bbff77f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:58.863573 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c55439-7112-48e5-9486-6afa20ba6f21 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:58.911820 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77e354-3382-4fcb-a421-52b3df40e77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:58.912755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77e354-3382-4fcb-a421-52b3df40e77e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:58.913055 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77e354-3382-4fcb-a421-52b3df40e77e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:58.918070 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77e354-3382-4fcb-a421-52b3df40e77e None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:58.918070 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77e354-3382-4fcb-a421-52b3df40e77e None admin] RBAC: Authorizing `identity:get_project(project_id=21519c13797a4d5b8858f4eed4d738f8)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:58.918070 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77e354-3382-4fcb-a421-52b3df40e77e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:58.918783 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 184/361] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:58 2022] GET /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:58.920106 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01aa97-7110-4b2f-bf4a-f248edca9e97 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:58.920392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01aa97-7110-4b2f-bf4a-f248edca9e97 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:58.920667 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01aa97-7110-4b2f-bf4a-f248edca9e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:58.922876 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:58.934466 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cabe7-9d72-4ccb-8cc7-f7a4bbff77f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:58.935942 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cabe7-9d72-4ccb-8cc7-f7a4bbff77f1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:58.941738 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3c01aa97-7110-4b2f-bf4a-f248edca9e97 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:24:58.942254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3c01aa97-7110-4b2f-bf4a-f248edca9e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:24:58.955478 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:59.068700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 185/362] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:24:59.069637 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01aa97-7110-4b2f-bf4a-f248edca9e97 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:59.091036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 178/363] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:24:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:24:59.091278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cabe7-9d72-4ccb-8cc7-f7a4bbff77f1 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:59.238074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:59.252148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cabe7-9d72-4ccb-8cc7-f7a4bbff77f1 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:59.295738 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01aa97-7110-4b2f-bf4a-f248edca9e97 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:59.320670 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367b6b0-fe7d-4f58-81d8-fc3005f7b3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:59.322100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367b6b0-fe7d-4f58-81d8-fc3005f7b3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:59.322592 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367b6b0-fe7d-4f58-81d8-fc3005f7b3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:59.323075 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367b6b0-fe7d-4f58-81d8-fc3005f7b3f0 None admin] PATH_INFO: `/v3/groups/640b54e919004efebb102e0e48da8ceb` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:59.327754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367b6b0-fe7d-4f58-81d8-fc3005f7b3f0 None admin] RBAC: Authorizing `identity:get_group(group_id=640b54e919004efebb102e0e48da8ceb)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:59.328137 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367b6b0-fe7d-4f58-81d8-fc3005f7b3f0 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:59.331362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 179/364] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 18 08:24:59 2022] GET /identity/v3/groups/640b54e919004efebb102e0e48da8ceb => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:59.332589 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367b6b0-fe7d-4f58-81d8-fc3005f7b3f0 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:59.352115 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b09e4-86ce-4c4f-aba6-1c3c1e276157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:59.353530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b09e4-86ce-4c4f-aba6-1c3c1e276157 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:59.353836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b09e4-86ce-4c4f-aba6-1c3c1e276157 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:59.354105 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b09e4-86ce-4c4f-aba6-1c3c1e276157 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:59.380155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b09e4-86ce-4c4f-aba6-1c3c1e276157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:59.395999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-569b09e4-86ce-4c4f-aba6-1c3c1e276157 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:24:59.395999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:59.395999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:59.395999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:59.395999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:59.395999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:59.395999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:59.395999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:59.395999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:59.395999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:59.395999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:59.395999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:59.395999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:59.395999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:59.395999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:59.395999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:59.397795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:59.397795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:59.397795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:59.397795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:59.397795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:59.397795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:59.397795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:59.397795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:59.397795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:59.397795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:59.397795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:59.397795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:59.397795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:59.397795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:59.397795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:59.397795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:59.399468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:59.399468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:59.399468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:59.399468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:59.399468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:59.399468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:59.399468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:59.399468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:59.399468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:59.399468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:59.399468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:24:59.399468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:59.399468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 186/365] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:24:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:59.399468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b09e4-86ce-4c4f-aba6-1c3c1e276157 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:59.408533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:59.567841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367b6b0-fe7d-4f58-81d8-fc3005f7b3f0 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:59.589260 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b09e4-86ce-4c4f-aba6-1c3c1e276157 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:59.645594 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3488cb1-9581-425d-93b9-7835500de1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:59.647638 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3488cb1-9581-425d-93b9-7835500de1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:59.650027 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b509ad1-e2d3-45d0-9212-8fca89a7a5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:59.650237 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3488cb1-9581-425d-93b9-7835500de1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:59.650557 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3488cb1-9581-425d-93b9-7835500de1e6 None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:59.651128 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b509ad1-e2d3-45d0-9212-8fca89a7a5ec None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:59.651407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b509ad1-e2d3-45d0-9212-8fca89a7a5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:59.651898 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b509ad1-e2d3-45d0-9212-8fca89a7a5ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:59.652916 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3488cb1-9581-425d-93b9-7835500de1e6 None admin] RBAC: Authorizing `identity:get_project(project_id=791daf53d0364b258d9cfb153ffa3111)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:59.655979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3488cb1-9581-425d-93b9-7835500de1e6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:59.659418 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 180/366] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:24:59 2022] GET /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111 => generated 298 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:59.661115 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3488cb1-9581-425d-93b9-7835500de1e6 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:59.674405 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b509ad1-e2d3-45d0-9212-8fca89a7a5ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:24:59.691119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-8b509ad1-e2d3-45d0-9212-8fca89a7a5ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:24:59.691119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:24:59.691119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:24:59.691119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:24:59.691119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:24:59.691119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:24:59.691119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:24:59.691119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:24:59.691119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:24:59.691119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:24:59.691119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:24:59.691119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:24:59.691119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:24:59.691119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:24:59.691119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:24:59.691119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:24:59.692849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:24:59.692849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:24:59.692849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:24:59.692849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:24:59.692849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:24:59.692849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:24:59.692849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:24:59.692849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:24:59.692849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:24:59.692849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:24:59.692849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:24:59.692849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:24:59.692849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:24:59.692849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:24:59.692849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:24:59.692849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:24:59.694345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:24:59.694345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:24:59.694345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:24:59.694345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:24:59.694345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:24:59.694345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:24:59.694345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:24:59.694345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:24:59.694345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:24:59.694345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:24:59.694345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:24:59.694345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:24:59.694345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 187/367] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:24:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:24:59.695629 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b509ad1-e2d3-45d0-9212-8fca89a7a5ec None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:59.709948 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:59.814320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3488cb1-9581-425d-93b9-7835500de1e6 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:59.838656 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:24:59.856514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccc091-3395-47b3-9c0b-aa3a2f1797b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:59.857221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccc091-3395-47b3-9c0b-aa3a2f1797b3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:59.857658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccc091-3395-47b3-9c0b-aa3a2f1797b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:59.857838 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccc091-3395-47b3-9c0b-aa3a2f1797b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:59.858679 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ccc091-3395-47b3-9c0b-aa3a2f1797b3 None admin] RBAC: Adding query filter params (group.id=640b54e919004efebb102e0e48da8ceb, role.id=587af95739954600bbf2411974601f74, scope.project.id=21519c13797a4d5b8858f4eed4d738f8) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:59.858991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ccc091-3395-47b3-9c0b-aa3a2f1797b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:59.860548 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ccc091-3395-47b3-9c0b-aa3a2f1797b3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:59.869131 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 181/368] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 18 08:24:59 2022] GET /identity/v3/role_assignments?group.id=640b54e919004efebb102e0e48da8ceb&scope.project.id=21519c13797a4d5b8858f4eed4d738f8&role.id=587af95739954600bbf2411974601f74 => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:59.870038 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccc091-3395-47b3-9c0b-aa3a2f1797b3 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:24:59.907893 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b509ad1-e2d3-45d0-9212-8fca89a7a5ec None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:24:59.963464 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66567a4e-23cd-41e1-b2d6-e8e887131d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:24:59.964602 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66567a4e-23cd-41e1-b2d6-e8e887131d66 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:24:59.965108 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66567a4e-23cd-41e1-b2d6-e8e887131d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:24:59.965587 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66567a4e-23cd-41e1-b2d6-e8e887131d66 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:24:59.966629 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66567a4e-23cd-41e1-b2d6-e8e887131d66 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:24:59.967166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66567a4e-23cd-41e1-b2d6-e8e887131d66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:24:59.969280 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66567a4e-23cd-41e1-b2d6-e8e887131d66 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:24:59.989977 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 188/369] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:24:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:24:59.992630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66567a4e-23cd-41e1-b2d6-e8e887131d66 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:00.066046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:00.080470 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccc091-3395-47b3-9c0b-aa3a2f1797b3 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:00.173939 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75020a-29b7-4ffc-b79f-52e17e43464d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:00.176042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75020a-29b7-4ffc-b79f-52e17e43464d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:00.176518 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75020a-29b7-4ffc-b79f-52e17e43464d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:00.177624 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75020a-29b7-4ffc-b79f-52e17e43464d None admin] PATH_INFO: `/v3/groups/640b54e919004efebb102e0e48da8ceb` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:00.181112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75020a-29b7-4ffc-b79f-52e17e43464d None admin] RBAC: Authorizing `identity:get_group(group_id=640b54e919004efebb102e0e48da8ceb)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:00.182979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75020a-29b7-4ffc-b79f-52e17e43464d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:00.185031 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 182/370] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 18 08:24:59 2022] GET /identity/v3/groups/640b54e919004efebb102e0e48da8ceb => generated 226 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:00.186081 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75020a-29b7-4ffc-b79f-52e17e43464d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:00.205454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66567a4e-23cd-41e1-b2d6-e8e887131d66 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:00.269995 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:00.295497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248362cb-5db9-44ed-9dfb-501a0d758b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:00.296713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248362cb-5db9-44ed-9dfb-501a0d758b27 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:00.297186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248362cb-5db9-44ed-9dfb-501a0d758b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:00.297673 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248362cb-5db9-44ed-9dfb-501a0d758b27 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:00.299096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248362cb-5db9-44ed-9dfb-501a0d758b27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:00.299626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248362cb-5db9-44ed-9dfb-501a0d758b27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:00.305486 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248362cb-5db9-44ed-9dfb-501a0d758b27 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:00.327372 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 189/371] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:24:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:00.328970 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248362cb-5db9-44ed-9dfb-501a0d758b27 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:00.342291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75020a-29b7-4ffc-b79f-52e17e43464d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:00.389451 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92476f-228f-43ef-b3fb-7687cbfe0c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:00.390184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92476f-228f-43ef-b3fb-7687cbfe0c48 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:00.390470 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92476f-228f-43ef-b3fb-7687cbfe0c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:00.390810 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92476f-228f-43ef-b3fb-7687cbfe0c48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:00.429755 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:00.476836 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92476f-228f-43ef-b3fb-7687cbfe0c48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:00.479636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92476f-228f-43ef-b3fb-7687cbfe0c48 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:00.543298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248362cb-5db9-44ed-9dfb-501a0d758b27 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:00.605330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 183/372] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:25:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:25:00.606933 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92476f-228f-43ef-b3fb-7687cbfe0c48 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:00.627443 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cbbe40-1d56-4092-b6bd-40e655f3202e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:00.628231 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbbe40-1d56-4092-b6bd-40e655f3202e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:00.628514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbbe40-1d56-4092-b6bd-40e655f3202e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:00.628788 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbbe40-1d56-4092-b6bd-40e655f3202e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:00.629860 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cbbe40-1d56-4092-b6bd-40e655f3202e None admin] RBAC: Adding query filter params (group.id=640b54e919004efebb102e0e48da8ceb, role.id=d2ba35e60f974010a5e7dd5c4bebecf8, scope.project.id=791daf53d0364b258d9cfb153ffa3111) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:00.630179 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cbbe40-1d56-4092-b6bd-40e655f3202e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:00.632088 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cbbe40-1d56-4092-b6bd-40e655f3202e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:00.649109 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 190/373] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 18 08:25:00 2022] GET /identity/v3/role_assignments?group.id=640b54e919004efebb102e0e48da8ceb&scope.project.id=791daf53d0364b258d9cfb153ffa3111&role.id=d2ba35e60f974010a5e7dd5c4bebecf8 => generated 260 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:00.649109 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cbbe40-1d56-4092-b6bd-40e655f3202e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:00.740075 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:00.772071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:00.785533 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92476f-228f-43ef-b3fb-7687cbfe0c48 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:00.850330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc54d21-f6bd-4770-a460-59a53882fef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:00.851477 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc54d21-f6bd-4770-a460-59a53882fef1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:00.852034 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc54d21-f6bd-4770-a460-59a53882fef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:00.853246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc54d21-f6bd-4770-a460-59a53882fef1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:00.891655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cbbe40-1d56-4092-b6bd-40e655f3202e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:00.939237 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a7e9a-de37-48dd-8f9b-97a36ae257d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:00.940255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a7e9a-de37-48dd-8f9b-97a36ae257d1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:00.940541 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a7e9a-de37-48dd-8f9b-97a36ae257d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:00.940816 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a7e9a-de37-48dd-8f9b-97a36ae257d1 None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:00.942143 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a7e9a-de37-48dd-8f9b-97a36ae257d1 None admin] RBAC: Authorizing `identity:get_project(project_id=21519c13797a4d5b8858f4eed4d738f8)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:00.943626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a7e9a-de37-48dd-8f9b-97a36ae257d1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:00.951235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 191/374] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:25:00 2022] GET /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8 => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:00.952807 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab35a9b-519d-433e-a640-d476b73d4c56 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:00.953094 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab35a9b-519d-433e-a640-d476b73d4c56 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:00.953378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab35a9b-519d-433e-a640-d476b73d4c56 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:00.955104 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 192/375] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:00.955944 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab35a9b-519d-433e-a640-d476b73d4c56 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:01.003228 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc54d21-f6bd-4770-a460-59a53882fef1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:01.004859 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc54d21-f6bd-4770-a460-59a53882fef1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:01.112098 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:01.201510 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab35a9b-519d-433e-a640-d476b73d4c56 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:01.205067 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 184/376] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:25:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:25:01.207622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f730d3-459c-4e9d-ab01-838dfbe47f5e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:01.208203 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f730d3-459c-4e9d-ab01-838dfbe47f5e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:01.208710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f730d3-459c-4e9d-ab01-838dfbe47f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:01.242190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:01.248001 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-f6f730d3-459c-4e9d-ab01-838dfbe47f5e None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:01.257739 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f6f730d3-459c-4e9d-ab01-838dfbe47f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:01.271193 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb00fc8-2d29-4e82-8f96-85e131277cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:01.273149 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb00fc8-2d29-4e82-8f96-85e131277cab None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:01.274134 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb00fc8-2d29-4e82-8f96-85e131277cab None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:01.275560 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb00fc8-2d29-4e82-8f96-85e131277cab None admin] PATH_INFO: `/v3/groups/640b54e919004efebb102e0e48da8ceb` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:01.278180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb00fc8-2d29-4e82-8f96-85e131277cab None admin] RBAC: Authorizing `identity:get_group(group_id=640b54e919004efebb102e0e48da8ceb)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:01.281340 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb00fc8-2d29-4e82-8f96-85e131277cab None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:01.284929 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 193/377] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 18 08:25:00 2022] GET /identity/v3/groups/640b54e919004efebb102e0e48da8ceb => generated 226 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:01.286619 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb00fc8-2d29-4e82-8f96-85e131277cab None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:01.389932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 185/378] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:01.391033 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f730d3-459c-4e9d-ab01-838dfbe47f5e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:01.437617 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb00fc8-2d29-4e82-8f96-85e131277cab None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:01.478230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5e6bf-f710-4a89-afdd-aa13f883dab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:01.479795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5e6bf-f710-4a89-afdd-aa13f883dab2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:01.480270 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5e6bf-f710-4a89-afdd-aa13f883dab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:01.480725 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5e6bf-f710-4a89-afdd-aa13f883dab2 None admin] PATH_INFO: `/v3/projects/eedba945f8d24629a712fbf156a2209f` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:01.483351 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5e6bf-f710-4a89-afdd-aa13f883dab2 None admin] RBAC: Authorizing `identity:get_project(project_id=eedba945f8d24629a712fbf156a2209f)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:01.485165 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5e6bf-f710-4a89-afdd-aa13f883dab2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:01.487267 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 194/379] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:25:01 2022] GET /identity/v3/projects/eedba945f8d24629a712fbf156a2209f => generated 344 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:01.488355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5e6bf-f710-4a89-afdd-aa13f883dab2 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:01.501815 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:01.512716 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f730d3-459c-4e9d-ab01-838dfbe47f5e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:01.559101 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:01.565208 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897144f1-7a2e-4ff4-a3ab-bca9e4426fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:01.565912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897144f1-7a2e-4ff4-a3ab-bca9e4426fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:01.566252 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897144f1-7a2e-4ff4-a3ab-bca9e4426fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:01.566509 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897144f1-7a2e-4ff4-a3ab-bca9e4426fa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:01.567345 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897144f1-7a2e-4ff4-a3ab-bca9e4426fa9 None admin] RBAC: Adding query filter params (group.id=640b54e919004efebb102e0e48da8ceb, role.id=d2ba35e60f974010a5e7dd5c4bebecf8, scope.project.id=21519c13797a4d5b8858f4eed4d738f8) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:01.567698 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897144f1-7a2e-4ff4-a3ab-bca9e4426fa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:01.579276 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897144f1-7a2e-4ff4-a3ab-bca9e4426fa9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:01.579276 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 186/380] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 18 08:25:01 2022] GET /identity/v3/role_assignments?group.id=640b54e919004efebb102e0e48da8ceb&scope.project.id=21519c13797a4d5b8858f4eed4d738f8&role.id=d2ba35e60f974010a5e7dd5c4bebecf8 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:01.580523 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897144f1-7a2e-4ff4-a3ab-bca9e4426fa9 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:01.685522 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5e6bf-f710-4a89-afdd-aa13f883dab2 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:01.794712 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532650d2-6d33-418a-9b84-f84b298284ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:01.795455 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532650d2-6d33-418a-9b84-f84b298284ad None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:01.795768 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532650d2-6d33-418a-9b84-f84b298284ad None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:01.796053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532650d2-6d33-418a-9b84-f84b298284ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:01.812810 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897144f1-7a2e-4ff4-a3ab-bca9e4426fa9 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:01.828191 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532650d2-6d33-418a-9b84-f84b298284ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:01.839622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-532650d2-6d33-418a-9b84-f84b298284ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:25:01.839622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:01.839622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:01.839622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:01.839622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:01.839622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:01.839622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:01.839622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:01.839622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:01.839622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:01.839622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:01.839622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:01.839622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:01.839622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:01.839622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:01.839622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:01.840583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:01.840583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:01.840583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:01.840583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:01.840583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:01.840583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:01.840583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:01.840583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:01.840583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:01.840583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:01.840583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:01.840583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:01.840583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:01.840583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:01.840583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:01.840583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:01.841465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:01.841465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:01.841465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:01.841465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:01.841465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:01.841465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:01.841465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:01.841465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:01.841465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:01.841465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:01.841465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:25:01.841465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:01.841465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 195/381] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:25:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:01.859384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532650d2-6d33-418a-9b84-f84b298284ad None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:01.888267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e15e2-c98c-4610-8a7e-00679685ec65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:01.889292 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e15e2-c98c-4610-8a7e-00679685ec65 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:01.889761 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e15e2-c98c-4610-8a7e-00679685ec65 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:01.890232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e15e2-c98c-4610-8a7e-00679685ec65 None admin] PATH_INFO: `/v3/groups/76f3265f09eb410fb1b3a78791fb8fef` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:01.900176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e15e2-c98c-4610-8a7e-00679685ec65 None admin] RBAC: Authorizing `identity:get_group(group_id=76f3265f09eb410fb1b3a78791fb8fef)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:01.901866 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e15e2-c98c-4610-8a7e-00679685ec65 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:01.906829 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 187/382] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 18 08:25:01 2022] GET /identity/v3/groups/76f3265f09eb410fb1b3a78791fb8fef => generated 229 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:01.920746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e15e2-c98c-4610-8a7e-00679685ec65 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:01.975117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:02.053804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532650d2-6d33-418a-9b84-f84b298284ad None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:02.098226 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:02.123577 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e15e2-c98c-4610-8a7e-00679685ec65 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:02.126804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ea2e4-4c7a-4a76-9106-130efc51f60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:02.127496 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ea2e4-4c7a-4a76-9106-130efc51f60a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:02.127815 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ea2e4-4c7a-4a76-9106-130efc51f60a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:02.128093 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ea2e4-4c7a-4a76-9106-130efc51f60a None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:02.128982 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ea2e4-4c7a-4a76-9106-130efc51f60a None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:02.129294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ea2e4-4c7a-4a76-9106-130efc51f60a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:02.130722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ea2e4-4c7a-4a76-9106-130efc51f60a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:02.141064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 196/383] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:25:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:25:02.147699 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ea2e4-4c7a-4a76-9106-130efc51f60a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:02.222701 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98db67-faa8-4ed8-856a-965f37e58e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:02.223893 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98db67-faa8-4ed8-856a-965f37e58e03 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:02.224408 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98db67-faa8-4ed8-856a-965f37e58e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:02.224747 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98db67-faa8-4ed8-856a-965f37e58e03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:02.225997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98db67-faa8-4ed8-856a-965f37e58e03 None admin] RBAC: Adding query filter params (group.id=76f3265f09eb410fb1b3a78791fb8fef, role.id=467d49b8a0d3435aaba58464cc8ef85c, scope.project.id=eedba945f8d24629a712fbf156a2209f) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:02.226335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98db67-faa8-4ed8-856a-965f37e58e03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:02.228766 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98db67-faa8-4ed8-856a-965f37e58e03 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:02.247096 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 188/384] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 18 08:25:01 2022] GET /identity/v3/role_assignments?group.id=76f3265f09eb410fb1b3a78791fb8fef&scope.project.id=eedba945f8d24629a712fbf156a2209f&role.id=467d49b8a0d3435aaba58464cc8ef85c => generated 260 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:02.392561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ea2e4-4c7a-4a76-9106-130efc51f60a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:02.498392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec75c58-c772-4c0a-9185-425dcaf4b77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:02.499898 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec75c58-c772-4c0a-9185-425dcaf4b77d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:02.500377 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec75c58-c772-4c0a-9185-425dcaf4b77d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:02.500841 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec75c58-c772-4c0a-9185-425dcaf4b77d None admin] PATH_INFO: `/v3/groups/640b54e919004efebb102e0e48da8ceb` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:02.509465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec75c58-c772-4c0a-9185-425dcaf4b77d None admin] RBAC: Authorizing `identity:get_group(group_id=640b54e919004efebb102e0e48da8ceb)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:02.511660 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec75c58-c772-4c0a-9185-425dcaf4b77d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:02.515180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 197/385] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 18 08:25:02 2022] GET /identity/v3/groups/640b54e919004efebb102e0e48da8ceb => generated 226 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:25:02.521123 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98db67-faa8-4ed8-856a-965f37e58e03 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:02.649167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:02.679225 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98db67-faa8-4ed8-856a-965f37e58e03 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:02.759888 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73de3c-64a3-4957-9de2-11355ac775f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:02.761545 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73de3c-64a3-4957-9de2-11355ac775f2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:02.762041 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73de3c-64a3-4957-9de2-11355ac775f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:02.762545 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73de3c-64a3-4957-9de2-11355ac775f2 None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:02.769171 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d73de3c-64a3-4957-9de2-11355ac775f2 None admin] RBAC: Authorizing `identity:get_project(project_id=791daf53d0364b258d9cfb153ffa3111)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:02.771234 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d73de3c-64a3-4957-9de2-11355ac775f2 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:02.773442 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 189/386] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:25:02 2022] GET /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:02.781259 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec75c58-c772-4c0a-9185-425dcaf4b77d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:02.885072 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:02.986970 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec75c58-c772-4c0a-9185-425dcaf4b77d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:03.071574 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d80e3-b05b-40d3-bb8c-8118076e21fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:03.072527 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d80e3-b05b-40d3-bb8c-8118076e21fa None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:03.072813 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d80e3-b05b-40d3-bb8c-8118076e21fa None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:03.073095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d80e3-b05b-40d3-bb8c-8118076e21fa None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111/groups/640b54e919004efebb102e0e48da8ceb/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:03.081450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d80e3-b05b-40d3-bb8c-8118076e21fa None admin] RBAC: Authorizing `identity:create_grant(project_id=791daf53d0364b258d9cfb153ffa3111, group_id=640b54e919004efebb102e0e48da8ceb, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:03.082994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d80e3-b05b-40d3-bb8c-8118076e21fa None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:03.195954 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 198/387] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Jul 18 08:25:02 2022] PUT /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111/groups/640b54e919004efebb102e0e48da8ceb/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:25:03.619628 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ac36f-3611-4dab-9524-6fe329b83116 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:03.620180 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ac36f-3611-4dab-9524-6fe329b83116 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:03.620293 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ac36f-3611-4dab-9524-6fe329b83116 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:03.622020 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 190/388] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:25:03.631124 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9a8b8-6669-4a72-8db1-82d354f65415 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:03.631862 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9a8b8-6669-4a72-8db1-82d354f65415 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:03.632332 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9a8b8-6669-4a72-8db1-82d354f65415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:03.673698 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b5d9a8b8-6669-4a72-8db1-82d354f65415 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:03.675983 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b5d9a8b8-6669-4a72-8db1-82d354f65415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:03.808205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bcd79-e6a9-4e92-bbe5-b64430050ab1 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:03.808702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bcd79-e6a9-4e92-bbe5-b64430050ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:03.809162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bcd79-e6a9-4e92-bbe5-b64430050ab1 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:03.811741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 191/389] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:25:03.817596 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cde909-8313-4c07-8d34-77fb9f42ddf6 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:03.818072 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cde909-8313-4c07-8d34-77fb9f42ddf6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:03.818531 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cde909-8313-4c07-8d34-77fb9f42ddf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:03.851732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-32cde909-8313-4c07-8d34-77fb9f42ddf6 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:03.851732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-32cde909-8313-4c07-8d34-77fb9f42ddf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:03.885397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 199/390] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:03.925946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9a8b8-6669-4a72-8db1-82d354f65415 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:03.996369 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 192/391] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:04.019808 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cde909-8313-4c07-8d34-77fb9f42ddf6 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:04.058023 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:04.245410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:04.270038 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9a8b8-6669-4a72-8db1-82d354f65415 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:04.286050 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cde909-8313-4c07-8d34-77fb9f42ddf6 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:04.356152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfa50e-1244-44e9-b515-1b1d89cd40c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:04.357430 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfa50e-1244-44e9-b515-1b1d89cd40c0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:04.358349 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfa50e-1244-44e9-b515-1b1d89cd40c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:04.358856 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfa50e-1244-44e9-b515-1b1d89cd40c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:04.368842 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a9e73-018c-480c-b3f1-2ff7de8105dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:04.370055 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a9e73-018c-480c-b3f1-2ff7de8105dc None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:04.370566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a9e73-018c-480c-b3f1-2ff7de8105dc None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:04.371045 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a9e73-018c-480c-b3f1-2ff7de8105dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:04.384955 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfa50e-1244-44e9-b515-1b1d89cd40c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:04.398342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f3bfa50e-1244-44e9-b515-1b1d89cd40c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:25:04.398342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:04.398342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:04.398342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:04.398342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:04.398342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:04.398342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:04.398342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:04.398342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:04.398342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:04.398342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:04.398342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:04.398342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:04.398342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:04.398342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:04.398342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:04.399978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:04.399978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:04.399978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:04.399978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:04.399978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:04.399978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:04.399978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:04.399978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:04.399978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:04.399978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:04.399978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:04.399978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:04.399978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:04.399978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:04.399978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:04.399978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:04.400784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:04.400784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:04.400784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:04.400784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:04.400784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:04.400784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:04.400784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:04.400784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:04.400784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:04.400784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:04.400784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:25:04.400784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:04.403687 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 200/392] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:25:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 479 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:25:04.404560 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a9e73-018c-480c-b3f1-2ff7de8105dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:04.413065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfa50e-1244-44e9-b515-1b1d89cd40c0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:04.416155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-832a9e73-018c-480c-b3f1-2ff7de8105dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:25:04.416155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:04.416155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:04.416155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:04.416155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:04.416155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:04.416155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:04.416155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:04.416155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:04.416155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:04.416155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:04.416155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:04.416155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:04.416155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:04.416155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:04.416155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:04.417076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:04.417076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:04.417076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:04.417076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:04.417076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:04.417076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:04.417076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:04.417076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:04.417076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:04.417076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:04.417076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:04.417076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:04.417076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:04.417076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:04.417076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:04.417076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:04.417875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:04.417875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:04.417875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:04.417875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:04.417875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:04.417875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:04.417875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:04.417875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:04.417875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:04.417875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:04.417875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:25:04.417875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:04.431082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 193/393] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:25:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:04.436646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a9e73-018c-480c-b3f1-2ff7de8105dc None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:04.554570 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:04.562728 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfa50e-1244-44e9-b515-1b1d89cd40c0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:04.607518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4f8a4-00be-47c9-9a97-4a17b01006d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:04.608242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4f8a4-00be-47c9-9a97-4a17b01006d6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:04.608531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4f8a4-00be-47c9-9a97-4a17b01006d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:04.608815 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4f8a4-00be-47c9-9a97-4a17b01006d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:04.609590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4f8a4-00be-47c9-9a97-4a17b01006d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:04.609905 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4f8a4-00be-47c9-9a97-4a17b01006d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:04.611313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4f8a4-00be-47c9-9a97-4a17b01006d6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:04.636019 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 201/394] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:25:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:25:04.638565 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c3546-57b8-4cac-a03a-9327cf5a11a8 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:04.639068 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c3546-57b8-4cac-a03a-9327cf5a11a8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:04.639563 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c3546-57b8-4cac-a03a-9327cf5a11a8 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:04.646748 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 202/395] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:04.648373 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c3546-57b8-4cac-a03a-9327cf5a11a8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:04.663835 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a9e73-018c-480c-b3f1-2ff7de8105dc None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:04.709040 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4ccd0-95d8-4af9-8649-ee68241378fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:04.710560 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4ccd0-95d8-4af9-8649-ee68241378fb None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:04.711253 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4ccd0-95d8-4af9-8649-ee68241378fb None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:04.712920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4ccd0-95d8-4af9-8649-ee68241378fb None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:04.714864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4ccd0-95d8-4af9-8649-ee68241378fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:04.715407 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4ccd0-95d8-4af9-8649-ee68241378fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:04.719632 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4ccd0-95d8-4af9-8649-ee68241378fb None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:04.741908 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 194/396] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:04.744501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57416e74-8152-4b2a-adc3-72f3d0d69bc4 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:04.744993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57416e74-8152-4b2a-adc3-72f3d0d69bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:04.745481 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57416e74-8152-4b2a-adc3-72f3d0d69bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:04.759740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:04.775124 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-57416e74-8152-4b2a-adc3-72f3d0d69bc4 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:04.776498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-57416e74-8152-4b2a-adc3-72f3d0d69bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:04.815312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c3546-57b8-4cac-a03a-9327cf5a11a8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:04.876931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60ca26-1a56-4f78-9804-43e49027b6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:04.878914 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60ca26-1a56-4f78-9804-43e49027b6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:04.879658 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60ca26-1a56-4f78-9804-43e49027b6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:04.880200 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60ca26-1a56-4f78-9804-43e49027b6c9 None admin] PATH_INFO: `/v3/groups/640b54e919004efebb102e0e48da8ceb` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:04.881700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60ca26-1a56-4f78-9804-43e49027b6c9 None admin] RBAC: Authorizing `identity:get_group(group_id=640b54e919004efebb102e0e48da8ceb)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:04.883945 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 195/397] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:04.884454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60ca26-1a56-4f78-9804-43e49027b6c9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:04.885049 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57416e74-8152-4b2a-adc3-72f3d0d69bc4 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:04.886648 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 203/398] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 18 08:25:04 2022] GET /identity/v3/groups/640b54e919004efebb102e0e48da8ceb => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:04.905160 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60ca26-1a56-4f78-9804-43e49027b6c9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:04.952365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:05.010211 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57416e74-8152-4b2a-adc3-72f3d0d69bc4 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:05.034521 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60ca26-1a56-4f78-9804-43e49027b6c9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:05.051544 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4ddde-8e92-42e4-9f20-3b3482073461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:05.052596 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4ddde-8e92-42e4-9f20-3b3482073461 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:05.052965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4ddde-8e92-42e4-9f20-3b3482073461 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:05.053162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4ddde-8e92-42e4-9f20-3b3482073461 None admin] PATH_INFO: `/v3/groups/640b54e919004efebb102e0e48da8ceb` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:05.054441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af4ddde-8e92-42e4-9f20-3b3482073461 None admin] RBAC: Authorizing `identity:get_group(group_id=640b54e919004efebb102e0e48da8ceb)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:05.056841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af4ddde-8e92-42e4-9f20-3b3482073461 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:05.063421 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 196/399] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 18 08:25:04 2022] GET /identity/v3/groups/640b54e919004efebb102e0e48da8ceb => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:05.063421 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4ddde-8e92-42e4-9f20-3b3482073461 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:05.063421 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:05.082985 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6f5e2-d00b-4693-a31c-e51c96c55216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:05.084686 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6f5e2-d00b-4693-a31c-e51c96c55216 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:05.084981 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6f5e2-d00b-4693-a31c-e51c96c55216 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:05.085266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6f5e2-d00b-4693-a31c-e51c96c55216 None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:05.088614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6f5e2-d00b-4693-a31c-e51c96c55216 None admin] RBAC: Authorizing `identity:get_project(project_id=21519c13797a4d5b8858f4eed4d738f8)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:05.091361 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6f5e2-d00b-4693-a31c-e51c96c55216 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:05.094073 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 204/400] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:25:04 2022] GET /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:05.095975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6f5e2-d00b-4693-a31c-e51c96c55216 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:05.189003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4ddde-8e92-42e4-9f20-3b3482073461 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:05.233283 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:05.235326 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9baee-efda-4568-b026-bd4084263d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:05.236965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9baee-efda-4568-b026-bd4084263d47 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:05.237493 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9baee-efda-4568-b026-bd4084263d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:05.238113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9baee-efda-4568-b026-bd4084263d47 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:05.242481 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6f5e2-d00b-4693-a31c-e51c96c55216 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:05.255139 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9baee-efda-4568-b026-bd4084263d47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:05.267162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-dce9baee-efda-4568-b026-bd4084263d47 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:25:05.267162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:05.267162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:05.267162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:05.267162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:05.267162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:05.267162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:05.267162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:05.267162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:05.267162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:05.267162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:05.267162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:05.267162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:05.267162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:05.267162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:05.267162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:05.268402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:05.268402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:05.268402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:05.268402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:05.268402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:05.268402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:05.268402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:05.268402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:05.268402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:05.268402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:05.268402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:05.268402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:05.268402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:05.268402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:05.268402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:05.268402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:05.269205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:05.269205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:05.269205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:05.269205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:05.269205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:05.269205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:05.269205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:05.269205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:05.269205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:05.269205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:05.269205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:25:05.269205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:05.273733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 197/401] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:25:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:05.275431 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9baee-efda-4568-b026-bd4084263d47 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:05.291465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7b686-c227-465c-a79f-fe61e1ef1044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:05.292914 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7b686-c227-465c-a79f-fe61e1ef1044 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:05.293763 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7b686-c227-465c-a79f-fe61e1ef1044 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:05.293763 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7b686-c227-465c-a79f-fe61e1ef1044 None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:05.295113 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7b686-c227-465c-a79f-fe61e1ef1044 None admin] RBAC: Authorizing `identity:get_project(project_id=791daf53d0364b258d9cfb153ffa3111)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:05.296920 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7b686-c227-465c-a79f-fe61e1ef1044 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:05.303719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 205/402] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:25:05 2022] GET /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:05.303719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7b686-c227-465c-a79f-fe61e1ef1044 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:05.369437 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:05.432551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9baee-efda-4568-b026-bd4084263d47 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:05.446552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7b686-c227-465c-a79f-fe61e1ef1044 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:05.489812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273be4a-656b-4f44-89c3-2f7138d5324b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:05.491580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273be4a-656b-4f44-89c3-2f7138d5324b None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:05.492187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273be4a-656b-4f44-89c3-2f7138d5324b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:05.492566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c01be1-363f-4d1e-997e-6aac7b145dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:05.492949 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273be4a-656b-4f44-89c3-2f7138d5324b None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8/groups/640b54e919004efebb102e0e48da8ceb/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:05.493265 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c01be1-363f-4d1e-997e-6aac7b145dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:05.493552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c01be1-363f-4d1e-997e-6aac7b145dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:05.494040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c01be1-363f-4d1e-997e-6aac7b145dc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:05.494968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c01be1-363f-4d1e-997e-6aac7b145dc2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:05.495290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c01be1-363f-4d1e-997e-6aac7b145dc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:05.496589 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273be4a-656b-4f44-89c3-2f7138d5324b None admin] RBAC: Authorizing `identity:create_grant(project_id=21519c13797a4d5b8858f4eed4d738f8, group_id=640b54e919004efebb102e0e48da8ceb, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:05.496922 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c01be1-363f-4d1e-997e-6aac7b145dc2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:05.498912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273be4a-656b-4f44-89c3-2f7138d5324b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:05.507750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 206/403] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:05.508660 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c01be1-363f-4d1e-997e-6aac7b145dc2 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:05.563079 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:05.588954 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 198/404] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Jul 18 08:25:05 2022] PUT /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8/groups/640b54e919004efebb102e0e48da8ceb/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:25:05.589595 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273be4a-656b-4f44-89c3-2f7138d5324b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:05.699106 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c01be1-363f-4d1e-997e-6aac7b145dc2 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:05.714741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273be4a-656b-4f44-89c3-2f7138d5324b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:05.748258 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:05.753245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0e880-ccd4-4ef5-b93a-5fbdcb3e3466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:05.754112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0e880-ccd4-4ef5-b93a-5fbdcb3e3466 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:05.754405 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0e880-ccd4-4ef5-b93a-5fbdcb3e3466 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:05.754690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0e880-ccd4-4ef5-b93a-5fbdcb3e3466 None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111/groups/640b54e919004efebb102e0e48da8ceb/roles/d2ba35e60f974010a5e7dd5c4bebecf8` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:05.757604 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0e880-ccd4-4ef5-b93a-5fbdcb3e3466 None admin] RBAC: Authorizing `identity:create_grant(project_id=791daf53d0364b258d9cfb153ffa3111, group_id=640b54e919004efebb102e0e48da8ceb, role_id=d2ba35e60f974010a5e7dd5c4bebecf8)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:05.759505 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0e880-ccd4-4ef5-b93a-5fbdcb3e3466 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:05.765736 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b6a96f-4c72-48d8-898e-cda52c533739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:05.766703 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6a96f-4c72-48d8-898e-cda52c533739 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:05.766990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6a96f-4c72-48d8-898e-cda52c533739 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:05.767328 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6a96f-4c72-48d8-898e-cda52c533739 None admin] PATH_INFO: `/v3/groups/640b54e919004efebb102e0e48da8ceb` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:05.768712 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b6a96f-4c72-48d8-898e-cda52c533739 None admin] RBAC: Authorizing `identity:get_group(group_id=640b54e919004efebb102e0e48da8ceb)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:05.770300 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b6a96f-4c72-48d8-898e-cda52c533739 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:05.776927 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 199/405] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 18 08:25:05 2022] GET /identity/v3/groups/640b54e919004efebb102e0e48da8ceb => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:05.776927 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfb7df-6138-40c9-a83d-01e67f2fb56d None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:05.776927 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfb7df-6138-40c9-a83d-01e67f2fb56d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:05.776927 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfb7df-6138-40c9-a83d-01e67f2fb56d None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:05.776927 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 200/406] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:05.777902 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfb7df-6138-40c9-a83d-01e67f2fb56d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:05.856132 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 207/407] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Jul 18 08:25:05 2022] PUT /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111/groups/640b54e919004efebb102e0e48da8ceb/roles/d2ba35e60f974010a5e7dd5c4bebecf8 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:25:05.857700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e289e8e-d95d-418e-8756-ec96d0ba855a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:05.860683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e289e8e-d95d-418e-8756-ec96d0ba855a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:05.861142 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e289e8e-d95d-418e-8756-ec96d0ba855a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:05.916949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7e289e8e-d95d-418e-8756-ec96d0ba855a None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:05.918247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7e289e8e-d95d-418e-8756-ec96d0ba855a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:05.964896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfb7df-6138-40c9-a83d-01e67f2fb56d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:06.070093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6dac6-fa6c-428c-a629-6740a32b855e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:06.072797 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6dac6-fa6c-428c-a629-6740a32b855e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:06.073593 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6dac6-fa6c-428c-a629-6740a32b855e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:06.074350 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6dac6-fa6c-428c-a629-6740a32b855e None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:06.076791 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6dac6-fa6c-428c-a629-6740a32b855e None admin] RBAC: Authorizing `identity:get_project(project_id=21519c13797a4d5b8858f4eed4d738f8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:06.078889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6dac6-fa6c-428c-a629-6740a32b855e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:06.080882 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 208/408] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:06.083706 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 201/409] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:25:05 2022] GET /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8 => generated 302 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:06.090229 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e289e8e-d95d-418e-8756-ec96d0ba855a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:06.109670 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6dac6-fa6c-428c-a629-6740a32b855e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:06.121965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:06.254208 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:06.328373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6dac6-fa6c-428c-a629-6740a32b855e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:06.332977 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e289e8e-d95d-418e-8756-ec96d0ba855a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:06.383016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e345c-b1ef-4341-aa1a-97d88c820283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:06.384494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e345c-b1ef-4341-aa1a-97d88c820283 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:06.384781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e345c-b1ef-4341-aa1a-97d88c820283 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:06.385063 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e345c-b1ef-4341-aa1a-97d88c820283 None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8/groups/640b54e919004efebb102e0e48da8ceb/roles/d2ba35e60f974010a5e7dd5c4bebecf8` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:06.387897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e345c-b1ef-4341-aa1a-97d88c820283 None admin] RBAC: Authorizing `identity:create_grant(project_id=21519c13797a4d5b8858f4eed4d738f8, group_id=640b54e919004efebb102e0e48da8ceb, role_id=d2ba35e60f974010a5e7dd5c4bebecf8)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:06.390138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e345c-b1ef-4341-aa1a-97d88c820283 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:06.397911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014cc46-5057-411f-8292-04dd0320e3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:06.399939 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014cc46-5057-411f-8292-04dd0320e3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:06.400432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014cc46-5057-411f-8292-04dd0320e3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:06.400927 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014cc46-5057-411f-8292-04dd0320e3d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:06.420986 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014cc46-5057-411f-8292-04dd0320e3d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:06.437979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-d014cc46-5057-411f-8292-04dd0320e3d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:25:06.437979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:06.437979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:06.437979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:06.437979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:06.437979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:06.437979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:06.437979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:06.437979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:06.437979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:06.437979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:06.437979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:06.437979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:06.437979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:06.437979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:06.437979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:06.453287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:06.453287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:06.453287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:06.453287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:06.453287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:06.453287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:06.453287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:06.453287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:06.453287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:06.453287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:06.453287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:06.453287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:06.453287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:06.453287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:06.453287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:06.453287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:06.454733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:06.454733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:06.454733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:06.454733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:06.454733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:06.454733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:06.454733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:06.454733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:06.454733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:06.454733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:06.454733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:25:06.454733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:06.454733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 202/410] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:25:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:06.454733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eddb20-e7db-4823-86a6-967444a619a9 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:06.456471 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eddb20-e7db-4823-86a6-967444a619a9 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:06.456471 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eddb20-e7db-4823-86a6-967444a619a9 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:06.456471 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 203/411] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:06.456471 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eddb20-e7db-4823-86a6-967444a619a9 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:06.515834 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 209/412] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Jul 18 08:25:06 2022] PUT /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8/groups/640b54e919004efebb102e0e48da8ceb/roles/d2ba35e60f974010a5e7dd5c4bebecf8 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:25:06.517715 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b306540-de46-4cae-bfb1-3ab8bbf10f72 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:06.518243 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b306540-de46-4cae-bfb1-3ab8bbf10f72 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:06.518747 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b306540-de46-4cae-bfb1-3ab8bbf10f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:06.553163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-6b306540-de46-4cae-bfb1-3ab8bbf10f72 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:06.554539 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6b306540-de46-4cae-bfb1-3ab8bbf10f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:06.640068 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eddb20-e7db-4823-86a6-967444a619a9 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:06.689444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:06.697259 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 210/413] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:06.728045 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b306540-de46-4cae-bfb1-3ab8bbf10f72 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:06.733279 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:06.735111 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390cb25-feb5-4dcc-ba1f-0996dee17a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:06.735914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390cb25-feb5-4dcc-ba1f-0996dee17a6a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:06.736204 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390cb25-feb5-4dcc-ba1f-0996dee17a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:06.736480 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390cb25-feb5-4dcc-ba1f-0996dee17a6a None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:06.737370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390cb25-feb5-4dcc-ba1f-0996dee17a6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:06.737686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390cb25-feb5-4dcc-ba1f-0996dee17a6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:06.739367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390cb25-feb5-4dcc-ba1f-0996dee17a6a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:06.751519 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 204/414] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:25:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:25:06.762686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390cb25-feb5-4dcc-ba1f-0996dee17a6a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:06.903623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b306540-de46-4cae-bfb1-3ab8bbf10f72 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:06.962643 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390cb25-feb5-4dcc-ba1f-0996dee17a6a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:06.969062 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113bb80-b574-4854-a911-8cab693a3bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:06.970590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113bb80-b574-4854-a911-8cab693a3bcc None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:06.971081 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113bb80-b574-4854-a911-8cab693a3bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:06.971605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113bb80-b574-4854-a911-8cab693a3bcc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:06.984143 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113bb80-b574-4854-a911-8cab693a3bcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:06.993497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-3113bb80-b574-4854-a911-8cab693a3bcc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:25:06.993497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:06.993497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:06.993497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:06.993497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:06.993497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:06.993497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:06.993497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:06.993497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:06.993497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:06.993497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:06.993497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:06.993497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:06.993497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:06.993497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:06.993497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:06.994444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:06.994444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:06.994444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:06.994444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:06.994444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:06.994444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:06.994444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:06.994444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:06.994444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:06.994444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:06.994444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:06.994444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:06.994444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:06.994444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:06.994444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:06.994444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:06.995250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:06.995250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:06.995250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:06.995250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:06.995250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:06.995250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:06.995250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:06.995250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:06.995250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:06.995250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:06.995250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:25:06.995250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:06.998130 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 211/415] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:25:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:25:07.009314 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113bb80-b574-4854-a911-8cab693a3bcc None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:07.026903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af668b-3a03-41d5-9a2a-d2973be21349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:07.028210 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af668b-3a03-41d5-9a2a-d2973be21349 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:07.028655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af668b-3a03-41d5-9a2a-d2973be21349 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:07.029136 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af668b-3a03-41d5-9a2a-d2973be21349 None admin] PATH_INFO: `/v3/groups/76f3265f09eb410fb1b3a78791fb8fef` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:07.030662 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af668b-3a03-41d5-9a2a-d2973be21349 None admin] RBAC: Authorizing `identity:get_group(group_id=76f3265f09eb410fb1b3a78791fb8fef)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:07.032360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af668b-3a03-41d5-9a2a-d2973be21349 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:07.035348 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 205/416] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 18 08:25:06 2022] GET /identity/v3/groups/76f3265f09eb410fb1b3a78791fb8fef => generated 229 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:07.042726 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af668b-3a03-41d5-9a2a-d2973be21349 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:07.156423 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:07.158391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:07.235714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af668b-3a03-41d5-9a2a-d2973be21349 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:07.239144 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113bb80-b574-4854-a911-8cab693a3bcc None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:07.277407 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7197d18-1ec2-4c1e-b214-73d481da479d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:07.278338 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7197d18-1ec2-4c1e-b214-73d481da479d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:07.278625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7197d18-1ec2-4c1e-b214-73d481da479d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:07.278901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7197d18-1ec2-4c1e-b214-73d481da479d None admin] PATH_INFO: `/v3/projects/eedba945f8d24629a712fbf156a2209f` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:07.280396 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7197d18-1ec2-4c1e-b214-73d481da479d None admin] RBAC: Authorizing `identity:get_project(project_id=eedba945f8d24629a712fbf156a2209f)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:07.281854 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7197d18-1ec2-4c1e-b214-73d481da479d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:07.292066 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 206/417] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:25:07 2022] GET /identity/v3/projects/eedba945f8d24629a712fbf156a2209f => generated 344 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:07.293604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7197d18-1ec2-4c1e-b214-73d481da479d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:07.322412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622d5d9-294e-4362-b9a6-796df193f7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:07.323408 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622d5d9-294e-4362-b9a6-796df193f7cd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:07.323899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622d5d9-294e-4362-b9a6-796df193f7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:07.324324 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622d5d9-294e-4362-b9a6-796df193f7cd None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:07.325240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622d5d9-294e-4362-b9a6-796df193f7cd None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:07.325719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622d5d9-294e-4362-b9a6-796df193f7cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:07.327143 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622d5d9-294e-4362-b9a6-796df193f7cd None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:07.347616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 212/418] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:25:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:07.355174 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622d5d9-294e-4362-b9a6-796df193f7cd None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:07.558837 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7197d18-1ec2-4c1e-b214-73d481da479d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:07.596762 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:07.635484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622d5d9-294e-4362-b9a6-796df193f7cd None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:07.638751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa956960-f82e-4513-9dd6-6a62c161e1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:07.639696 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa956960-f82e-4513-9dd6-6a62c161e1f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:07.639999 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa956960-f82e-4513-9dd6-6a62c161e1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:07.640278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa956960-f82e-4513-9dd6-6a62c161e1f3 None admin] PATH_INFO: `/v3/projects/eedba945f8d24629a712fbf156a2209f/groups/76f3265f09eb410fb1b3a78791fb8fef/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:07.649623 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa956960-f82e-4513-9dd6-6a62c161e1f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=eedba945f8d24629a712fbf156a2209f, group_id=76f3265f09eb410fb1b3a78791fb8fef, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:07.651037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa956960-f82e-4513-9dd6-6a62c161e1f3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:07.677007 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:07.687733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc9abe-c782-41a5-8950-202c67e36540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:07.688417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc9abe-c782-41a5-8950-202c67e36540 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:07.688706 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc9abe-c782-41a5-8950-202c67e36540 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:07.688983 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc9abe-c782-41a5-8950-202c67e36540 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:07.753426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 207/419] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Jul 18 08:25:07 2022] PUT /identity/v3/projects/eedba945f8d24629a712fbf156a2209f/groups/76f3265f09eb410fb1b3a78791fb8fef/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:25:07.797951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc9abe-c782-41a5-8950-202c67e36540 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:07.799569 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc9abe-c782-41a5-8950-202c67e36540 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:07.934605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 213/420] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:25:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 08:25:07.943265 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa956960-f82e-4513-9dd6-6a62c161e1f3 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:08.193380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa956960-f82e-4513-9dd6-6a62c161e1f3 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:08.262719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d06c0-1fde-44d8-8d68-03beb2739cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:08.263353 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80140735-713f-405c-a7c8-49096ed63f36 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:08.264025 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80140735-713f-405c-a7c8-49096ed63f36 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:08.264025 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80140735-713f-405c-a7c8-49096ed63f36 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:08.265571 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d06c0-1fde-44d8-8d68-03beb2739cae None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:08.265970 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 214/421] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:08.266487 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d06c0-1fde-44d8-8d68-03beb2739cae None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:08.267144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d06c0-1fde-44d8-8d68-03beb2739cae None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:08.269746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d06c0-1fde-44d8-8d68-03beb2739cae None admin] RBAC: Authorizing `identity:get_project(project_id=791daf53d0364b258d9cfb153ffa3111)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:08.271961 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d06c0-1fde-44d8-8d68-03beb2739cae None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:08.273094 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e0f93-c6f4-4e9e-9c28-a6c64d19131b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:08.273514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e0f93-c6f4-4e9e-9c28-a6c64d19131b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:08.274768 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e0f93-c6f4-4e9e-9c28-a6c64d19131b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:08.277861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 208/422] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:25:07 2022] GET /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111 => generated 298 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:08.286563 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d06c0-1fde-44d8-8d68-03beb2739cae None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:08.295756 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f19e0f93-c6f4-4e9e-9c28-a6c64d19131b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:08.298122 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f19e0f93-c6f4-4e9e-9c28-a6c64d19131b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:08.325236 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:08.377373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:08.391438 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 215/423] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:25:08.427571 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e0f93-c6f4-4e9e-9c28-a6c64d19131b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:08.495003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d06c0-1fde-44d8-8d68-03beb2739cae None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:08.548631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371f2e8-cc92-4026-87d0-5ca90d5dbb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:08.549978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371f2e8-cc92-4026-87d0-5ca90d5dbb09 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:08.550475 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371f2e8-cc92-4026-87d0-5ca90d5dbb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:08.550926 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371f2e8-cc92-4026-87d0-5ca90d5dbb09 None admin] PATH_INFO: `/v3/groups/640b54e919004efebb102e0e48da8ceb` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:08.552450 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371f2e8-cc92-4026-87d0-5ca90d5dbb09 None admin] RBAC: Authorizing `identity:get_group(group_id=640b54e919004efebb102e0e48da8ceb)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:08.554090 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371f2e8-cc92-4026-87d0-5ca90d5dbb09 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:08.556983 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 209/424] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 18 08:25:08 2022] GET /identity/v3/groups/640b54e919004efebb102e0e48da8ceb => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:08.562951 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371f2e8-cc92-4026-87d0-5ca90d5dbb09 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:08.679045 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e0f93-c6f4-4e9e-9c28-a6c64d19131b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:08.721572 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ed9ce-4940-4167-8806-154204ac3377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:08.722402 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ed9ce-4940-4167-8806-154204ac3377 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:08.722690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ed9ce-4940-4167-8806-154204ac3377 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:08.722968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ed9ce-4940-4167-8806-154204ac3377 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:08.737310 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ed9ce-4940-4167-8806-154204ac3377 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:08.745925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-602ed9ce-4940-4167-8806-154204ac3377 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:25:08.745925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:08.745925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:08.745925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:08.745925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:08.745925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:08.745925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:08.745925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:08.745925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:08.745925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:08.745925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:08.745925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:08.745925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:08.745925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:08.745925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:08.745925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:08.746879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:08.746879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:08.746879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:08.746879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:08.746879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:08.746879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:08.746879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:08.746879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:08.746879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:08.746879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:08.746879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:08.746879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:08.746879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:08.746879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:08.746879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:08.746879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:08.747722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:08.747722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:08.747722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:08.747722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:08.747722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:08.747722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:08.747722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:08.747722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:08.747722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:08.747722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:08.747722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 08:25:08.747722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:08.747722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 216/425] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:25:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:08.753516 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ed9ce-4940-4167-8806-154204ac3377 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:08.756676 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371f2e8-cc92-4026-87d0-5ca90d5dbb09 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:08.774166 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:08.775507 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:08.801183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5eec4-3d2e-4853-a753-b1e98e15a1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:08.802336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5eec4-3d2e-4853-a753-b1e98e15a1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:08.802857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5eec4-3d2e-4853-a753-b1e98e15a1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:08.803335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5eec4-3d2e-4853-a753-b1e98e15a1b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:08.804497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5eec4-3d2e-4853-a753-b1e98e15a1b5 None admin] RBAC: Adding query filter params (group.id=640b54e919004efebb102e0e48da8ceb, role.id=587af95739954600bbf2411974601f74, scope.project.id=791daf53d0364b258d9cfb153ffa3111) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:08.805599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5eec4-3d2e-4853-a753-b1e98e15a1b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:08.807531 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5eec4-3d2e-4853-a753-b1e98e15a1b5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:08.823796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 210/426] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 18 08:25:08 2022] GET /identity/v3/role_assignments?group.id=640b54e919004efebb102e0e48da8ceb&scope.project.id=791daf53d0364b258d9cfb153ffa3111&role.id=587af95739954600bbf2411974601f74 => generated 611 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:08.884375 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ed9ce-4940-4167-8806-154204ac3377 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:08.924741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc57e1-dec7-44bd-976b-44561fa844ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:08.925443 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc57e1-dec7-44bd-976b-44561fa844ba None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:08.925734 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc57e1-dec7-44bd-976b-44561fa844ba None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:08.926018 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc57e1-dec7-44bd-976b-44561fa844ba None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:08.926813 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc57e1-dec7-44bd-976b-44561fa844ba None admin] RBAC: Adding query filter params (name=member) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:08.927128 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc57e1-dec7-44bd-976b-44561fa844ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:08.928627 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc57e1-dec7-44bd-976b-44561fa844ba None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:08.940223 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 217/427] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:25:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:08.946295 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5eec4-3d2e-4853-a753-b1e98e15a1b5 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:09.083681 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5eec4-3d2e-4853-a753-b1e98e15a1b5 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:09.129385 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c360d-2871-4d59-97e7-03f1d4be9b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:09.130065 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c360d-2871-4d59-97e7-03f1d4be9b91 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:09.130352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c360d-2871-4d59-97e7-03f1d4be9b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:09.130629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c360d-2871-4d59-97e7-03f1d4be9b91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:09.153350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de117168-c0d4-40c4-87a0-ab6b73d87c85 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:09.153621 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de117168-c0d4-40c4-87a0-ab6b73d87c85 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:09.153943 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de117168-c0d4-40c4-87a0-ab6b73d87c85 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:09.155867 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 218/428] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:09.165135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4ce27-9e72-435b-be28-5ed08aabb683 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:09.165553 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4ce27-9e72-435b-be28-5ed08aabb683 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:09.165704 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4ce27-9e72-435b-be28-5ed08aabb683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:09.204001 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1dd4ce27-9e72-435b-be28-5ed08aabb683 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:09.205236 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1dd4ce27-9e72-435b-be28-5ed08aabb683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:09.207564 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:09.249691 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c360d-2871-4d59-97e7-03f1d4be9b91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:09.251193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c360d-2871-4d59-97e7-03f1d4be9b91 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:09.364708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 219/429] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:09.377444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 211/430] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:25:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:25:09.384512 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4ce27-9e72-435b-be28-5ed08aabb683 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:09.399715 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c360d-2871-4d59-97e7-03f1d4be9b91 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:09.420082 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:09.494756 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:09.520364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c360d-2871-4d59-97e7-03f1d4be9b91 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:09.563089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c2258-73f3-4098-b3f3-4394e1d7d5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:09.564058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c2258-73f3-4098-b3f3-4394e1d7d5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:09.564348 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c2258-73f3-4098-b3f3-4394e1d7d5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:09.564627 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c2258-73f3-4098-b3f3-4394e1d7d5b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:09.578005 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c2258-73f3-4098-b3f3-4394e1d7d5b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:09.588378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-873c2258-73f3-4098-b3f3-4394e1d7d5b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:25:09.588378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:09.588378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:09.588378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:09.588378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:09.588378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:09.588378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:09.588378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:09.588378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:09.588378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:09.588378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:09.588378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:09.588378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:09.588378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:09.588378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:09.588378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:09.589340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:09.589340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:09.589340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:09.589340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:09.589340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:09.589340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:09.589340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:09.589340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:09.589340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:09.589340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:09.589340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:09.589340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:09.589340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:09.589340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:09.589340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:09.589340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:09.590155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:09.590155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:09.590155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:09.590155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:09.590155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:09.590155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:09.590155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:09.590155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:09.590155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:09.590155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:09.590155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:25:09.590155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:09.590155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 212/431] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:25:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:25:09.591896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b1724-ec83-4a0b-b5a7-7ab279ca4299 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:09.591896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b1724-ec83-4a0b-b5a7-7ab279ca4299 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:09.596398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b1724-ec83-4a0b-b5a7-7ab279ca4299 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:09.601594 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 213/432] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:09 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:09.602916 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b1724-ec83-4a0b-b5a7-7ab279ca4299 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:09.617437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4ce27-9e72-435b-be28-5ed08aabb683 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:09.686761 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072b83b-806c-4d4e-b0b3-5714e86a3f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:09.687883 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072b83b-806c-4d4e-b0b3-5714e86a3f74 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:09.688242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072b83b-806c-4d4e-b0b3-5714e86a3f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:09.688595 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072b83b-806c-4d4e-b0b3-5714e86a3f74 None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:09.690562 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072b83b-806c-4d4e-b0b3-5714e86a3f74 None admin] RBAC: Authorizing `identity:get_project(project_id=21519c13797a4d5b8858f4eed4d738f8)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:09.691549 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072b83b-806c-4d4e-b0b3-5714e86a3f74 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:09.698319 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 220/433] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:25:09 2022] GET /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8 => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:09.698319 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed341ca2-7ec8-43c9-ab8a-a6c9a6bb6e2e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:09.698319 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed341ca2-7ec8-43c9-ab8a-a6c9a6bb6e2e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:09.698319 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed341ca2-7ec8-43c9-ab8a-a6c9a6bb6e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:09.717890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ed341ca2-7ec8-43c9-ab8a-a6c9a6bb6e2e None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:09.718846 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ed341ca2-7ec8-43c9-ab8a-a6c9a6bb6e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:09.773078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b1724-ec83-4a0b-b5a7-7ab279ca4299 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:09.814457 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3c7bd-fb58-4b10-b74c-53d7122717a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:09.815138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3c7bd-fb58-4b10-b74c-53d7122717a7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:09.815424 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3c7bd-fb58-4b10-b74c-53d7122717a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:09.815739 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3c7bd-fb58-4b10-b74c-53d7122717a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:09.816495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3c7bd-fb58-4b10-b74c-53d7122717a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:09.816802 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3c7bd-fb58-4b10-b74c-53d7122717a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:09.818194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3c7bd-fb58-4b10-b74c-53d7122717a7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:09.822161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:09.839955 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 214/434] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:09.841469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3c7bd-fb58-4b10-b74c-53d7122717a7 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:09.856011 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 221/435] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:09.858532 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed341ca2-7ec8-43c9-ab8a-a6c9a6bb6e2e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:09.874100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:09.992010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3c7bd-fb58-4b10-b74c-53d7122717a7 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:10.025896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed341ca2-7ec8-43c9-ab8a-a6c9a6bb6e2e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:10.034927 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be21d7-edc9-406e-a114-5968ab8caa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:10.035846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be21d7-edc9-406e-a114-5968ab8caa8b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:10.036131 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be21d7-edc9-406e-a114-5968ab8caa8b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:10.036405 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be21d7-edc9-406e-a114-5968ab8caa8b None admin] PATH_INFO: `/v3/groups/640b54e919004efebb102e0e48da8ceb` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:10.037664 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2be21d7-edc9-406e-a114-5968ab8caa8b None admin] RBAC: Authorizing `identity:get_group(group_id=640b54e919004efebb102e0e48da8ceb)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:10.039114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2be21d7-edc9-406e-a114-5968ab8caa8b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:10.041159 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 215/436] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 18 08:25:09 2022] GET /identity/v3/groups/640b54e919004efebb102e0e48da8ceb => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:10.041656 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be21d7-edc9-406e-a114-5968ab8caa8b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:10.089567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df333f-e145-49fd-a8e7-088dda14cc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:10.090244 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df333f-e145-49fd-a8e7-088dda14cc92 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:10.090617 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df333f-e145-49fd-a8e7-088dda14cc92 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:10.090854 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df333f-e145-49fd-a8e7-088dda14cc92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:10.165735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:10.180854 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df333f-e145-49fd-a8e7-088dda14cc92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:10.182374 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df333f-e145-49fd-a8e7-088dda14cc92 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:10.192110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:10.236964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be21d7-edc9-406e-a114-5968ab8caa8b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:10.302995 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94805852-5311-4bef-8d13-d916cf983670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:10.304435 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94805852-5311-4bef-8d13-d916cf983670 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:10.305077 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94805852-5311-4bef-8d13-d916cf983670 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:10.305577 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94805852-5311-4bef-8d13-d916cf983670 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:10.324945 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94805852-5311-4bef-8d13-d916cf983670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:10.341535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-94805852-5311-4bef-8d13-d916cf983670 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:25:10.341535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:10.341535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:10.341535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:10.341535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:10.341535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:10.341535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:10.341535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:10.341535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:10.341535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:10.341535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:10.341535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:10.341535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:10.341535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:10.341535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:10.341535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:10.342700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:10.342700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:10.342700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:10.342700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:10.342700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:10.342700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:10.342700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:10.342700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:10.342700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:10.342700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:10.342700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:10.342700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:10.342700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:10.342700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:10.342700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:10.342700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:10.343892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:10.343892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:10.343892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:10.343892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:10.343892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:10.343892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:10.343892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:10.343892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:10.343892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:10.343892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:10.343892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:25:10.343892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:10.344549 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 216/437] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:25:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:10.345597 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94805852-5311-4bef-8d13-d916cf983670 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:10.361633 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 222/438] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:25:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:25:10.361633 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df333f-e145-49fd-a8e7-088dda14cc92 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:10.529618 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94805852-5311-4bef-8d13-d916cf983670 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:10.568234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df333f-e145-49fd-a8e7-088dda14cc92 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:10.577349 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:10.584401 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:10.593844 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3f119-33bf-4a12-8137-3c3868ba075e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:10.594958 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3f119-33bf-4a12-8137-3c3868ba075e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:10.595602 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3f119-33bf-4a12-8137-3c3868ba075e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:10.596324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3f119-33bf-4a12-8137-3c3868ba075e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:10.597979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3f119-33bf-4a12-8137-3c3868ba075e None admin] RBAC: Adding query filter params (group.id=640b54e919004efebb102e0e48da8ceb, role.id=587af95739954600bbf2411974601f74, scope.project.id=21519c13797a4d5b8858f4eed4d738f8) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:10.598543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3f119-33bf-4a12-8137-3c3868ba075e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:10.601329 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3f119-33bf-4a12-8137-3c3868ba075e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:10.613487 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 217/439] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 18 08:25:10 2022] GET /identity/v3/role_assignments?group.id=640b54e919004efebb102e0e48da8ceb&scope.project.id=21519c13797a4d5b8858f4eed4d738f8&role.id=587af95739954600bbf2411974601f74 => generated 611 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:10.620429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3f119-33bf-4a12-8137-3c3868ba075e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:10.632357 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1589eb0-7fb5-4bf8-8a08-bf1bd0aa17c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:10.634315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1589eb0-7fb5-4bf8-8a08-bf1bd0aa17c5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:10.634646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1589eb0-7fb5-4bf8-8a08-bf1bd0aa17c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:10.634872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1589eb0-7fb5-4bf8-8a08-bf1bd0aa17c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:10.635743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1589eb0-7fb5-4bf8-8a08-bf1bd0aa17c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:10.636222 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1589eb0-7fb5-4bf8-8a08-bf1bd0aa17c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:10.638157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1589eb0-7fb5-4bf8-8a08-bf1bd0aa17c5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:10.650516 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 223/440] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:10.652069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dab6f-cc24-47f5-8f52-7f0c76706e68 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:10.652372 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dab6f-cc24-47f5-8f52-7f0c76706e68 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:10.652618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dab6f-cc24-47f5-8f52-7f0c76706e68 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:10.654382 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 224/441] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:10.660016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dab6f-cc24-47f5-8f52-7f0c76706e68 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:10.814095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dab6f-cc24-47f5-8f52-7f0c76706e68 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:10.822955 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3f119-33bf-4a12-8137-3c3868ba075e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:10.868790 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb2935-f7a1-4505-a71e-6362ea0c2a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:10.870959 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb2935-f7a1-4505-a71e-6362ea0c2a0f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:10.871268 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb2935-f7a1-4505-a71e-6362ea0c2a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:10.871552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb2935-f7a1-4505-a71e-6362ea0c2a0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:10.897304 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7aff4-98fd-4392-863f-844798c581f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:10.898840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7aff4-98fd-4392-863f-844798c581f7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:10.899322 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7aff4-98fd-4392-863f-844798c581f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:10.900102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7aff4-98fd-4392-863f-844798c581f7 None admin] PATH_INFO: `/v3/projects/791daf53d0364b258d9cfb153ffa3111` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:10.902285 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc7aff4-98fd-4392-863f-844798c581f7 None admin] RBAC: Authorizing `identity:get_project(project_id=791daf53d0364b258d9cfb153ffa3111)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:10.905166 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc7aff4-98fd-4392-863f-844798c581f7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:10.908449 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 218/442] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:25:10 2022] GET /identity/v3/projects/791daf53d0364b258d9cfb153ffa3111 => generated 298 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:10.910627 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241e143-4ee8-4fc9-9d2a-06ecf7145fbb None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:10.911122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241e143-4ee8-4fc9-9d2a-06ecf7145fbb None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:10.911593 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241e143-4ee8-4fc9-9d2a-06ecf7145fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:10.917736 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:10.941183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-8241e143-4ee8-4fc9-9d2a-06ecf7145fbb None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:10.943268 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8241e143-4ee8-4fc9-9d2a-06ecf7145fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:10.955085 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb2935-f7a1-4505-a71e-6362ea0c2a0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:10.956617 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb2935-f7a1-4505-a71e-6362ea0c2a0f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:11.027250 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:11.056398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 219/443] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:11.056398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241e143-4ee8-4fc9-9d2a-06ecf7145fbb None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:11.114459 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 225/444] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:25:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:25:11.122478 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb2935-f7a1-4505-a71e-6362ea0c2a0f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:11.201441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241e143-4ee8-4fc9-9d2a-06ecf7145fbb None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:11.215419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:11.241594 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77a117-ab43-4e02-9bb6-da466c5ccead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:11.242719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77a117-ab43-4e02-9bb6-da466c5ccead None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:11.243068 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77a117-ab43-4e02-9bb6-da466c5ccead None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:11.243354 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77a117-ab43-4e02-9bb6-da466c5ccead None admin] PATH_INFO: `/v3/groups/640b54e919004efebb102e0e48da8ceb` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:11.244660 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77a117-ab43-4e02-9bb6-da466c5ccead None admin] RBAC: Authorizing `identity:get_group(group_id=640b54e919004efebb102e0e48da8ceb)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:11.246217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77a117-ab43-4e02-9bb6-da466c5ccead None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:11.247941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 220/445] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 18 08:25:11 2022] GET /identity/v3/groups/640b54e919004efebb102e0e48da8ceb => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:11.248765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77a117-ab43-4e02-9bb6-da466c5ccead None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:11.263142 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb2935-f7a1-4505-a71e-6362ea0c2a0f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:11.303948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537b8b1-d373-4421-b05a-ea1eb5e9372c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:11.305405 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537b8b1-d373-4421-b05a-ea1eb5e9372c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:11.306424 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537b8b1-d373-4421-b05a-ea1eb5e9372c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:11.306717 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537b8b1-d373-4421-b05a-ea1eb5e9372c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:11.317690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5537b8b1-d373-4421-b05a-ea1eb5e9372c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:11.326700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-5537b8b1-d373-4421-b05a-ea1eb5e9372c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:25:11.326700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:11.326700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:11.326700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:11.326700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:11.326700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:11.326700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:11.326700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:11.326700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:11.326700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:11.326700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:11.326700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:11.326700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:11.326700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:11.326700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:11.326700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:11.328592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:11.328592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:11.328592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:11.328592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:11.328592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:11.328592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:11.328592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:11.328592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:11.328592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:11.328592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:11.328592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:11.328592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:11.328592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:11.328592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:11.328592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:11.328592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:11.330239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:11.330239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:11.330239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:11.330239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:11.330239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:11.330239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:11.330239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:11.330239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:11.330239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:11.330239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:11.330239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:25:11.330239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:11.330239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 226/446] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:25:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:11.330239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537b8b1-d373-4421-b05a-ea1eb5e9372c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:11.340329 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:11.397490 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77a117-ab43-4e02-9bb6-da466c5ccead None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:11.442196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694ae8d-d1c3-4f48-b371-7a03d609592e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:11.443135 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694ae8d-d1c3-4f48-b371-7a03d609592e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:11.443418 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694ae8d-d1c3-4f48-b371-7a03d609592e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:11.443720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694ae8d-d1c3-4f48-b371-7a03d609592e None admin] PATH_INFO: `/v3/projects/21519c13797a4d5b8858f4eed4d738f8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:11.445107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3694ae8d-d1c3-4f48-b371-7a03d609592e None admin] RBAC: Authorizing `identity:get_project(project_id=21519c13797a4d5b8858f4eed4d738f8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:11.447206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3694ae8d-d1c3-4f48-b371-7a03d609592e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:11.449406 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 221/447] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:25:11 2022] GET /identity/v3/projects/21519c13797a4d5b8858f4eed4d738f8 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:11.450029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694ae8d-d1c3-4f48-b371-7a03d609592e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:11.475497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537b8b1-d373-4421-b05a-ea1eb5e9372c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:11.526742 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:11.541750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fcc7b-8edf-4206-9e80-928e04c4f9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:11.542930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fcc7b-8edf-4206-9e80-928e04c4f9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:11.543404 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fcc7b-8edf-4206-9e80-928e04c4f9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:11.543888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fcc7b-8edf-4206-9e80-928e04c4f9b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:11.545528 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8fcc7b-8edf-4206-9e80-928e04c4f9b5 None admin] RBAC: Adding query filter params (group.id=640b54e919004efebb102e0e48da8ceb, role.id=d2ba35e60f974010a5e7dd5c4bebecf8, scope.project.id=791daf53d0364b258d9cfb153ffa3111) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:11.546082 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8fcc7b-8edf-4206-9e80-928e04c4f9b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:11.549126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8fcc7b-8edf-4206-9e80-928e04c4f9b5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:11.569777 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 227/448] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 18 08:25:11 2022] GET /identity/v3/role_assignments?group.id=640b54e919004efebb102e0e48da8ceb&scope.project.id=791daf53d0364b258d9cfb153ffa3111&role.id=d2ba35e60f974010a5e7dd5c4bebecf8 => generated 611 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:11.569777 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fcc7b-8edf-4206-9e80-928e04c4f9b5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:11.591612 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694ae8d-d1c3-4f48-b371-7a03d609592e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:11.645736 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:11.668112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d986e5-90c5-482d-a807-f8630fefab44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:11.669173 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d986e5-90c5-482d-a807-f8630fefab44 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:11.669686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d986e5-90c5-482d-a807-f8630fefab44 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:11.670155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d986e5-90c5-482d-a807-f8630fefab44 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:11.671654 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d986e5-90c5-482d-a807-f8630fefab44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:11.673303 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d986e5-90c5-482d-a807-f8630fefab44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:11.675173 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d986e5-90c5-482d-a807-f8630fefab44 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:11.698265 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 222/449] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:25:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:11.701691 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d986e5-90c5-482d-a807-f8630fefab44 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:11.734535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fcc7b-8edf-4206-9e80-928e04c4f9b5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:11.797435 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f16542-7983-4572-bbe5-9870a566b2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:11.798876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f16542-7983-4572-bbe5-9870a566b2bf None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:11.799356 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f16542-7983-4572-bbe5-9870a566b2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:11.799843 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f16542-7983-4572-bbe5-9870a566b2bf None admin] PATH_INFO: `/v3/groups/640b54e919004efebb102e0e48da8ceb` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:11.801972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f16542-7983-4572-bbe5-9870a566b2bf None admin] RBAC: Authorizing `identity:get_group(group_id=640b54e919004efebb102e0e48da8ceb)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:11.804414 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f16542-7983-4572-bbe5-9870a566b2bf None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:11.806848 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 228/450] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 18 08:25:11 2022] GET /identity/v3/groups/640b54e919004efebb102e0e48da8ceb => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:11.813680 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f16542-7983-4572-bbe5-9870a566b2bf None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:11.885109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d986e5-90c5-482d-a807-f8630fefab44 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:11.925205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:11.946590 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d36513-8c27-4e10-8955-89ececc53685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:11.952753 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d36513-8c27-4e10-8955-89ececc53685 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:11.952753 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d36513-8c27-4e10-8955-89ececc53685 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:11.952753 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d36513-8c27-4e10-8955-89ececc53685 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:11.993309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f16542-7983-4572-bbe5-9870a566b2bf None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:12.044282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:12.049066 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1703fb-cd80-40e0-b0ec-57368701b615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:12.049937 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1703fb-cd80-40e0-b0ec-57368701b615 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:12.050351 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1703fb-cd80-40e0-b0ec-57368701b615 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:12.050734 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1703fb-cd80-40e0-b0ec-57368701b615 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:12.052535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1703fb-cd80-40e0-b0ec-57368701b615 None admin] RBAC: Adding query filter params (group.id=640b54e919004efebb102e0e48da8ceb, role.id=d2ba35e60f974010a5e7dd5c4bebecf8, scope.project.id=21519c13797a4d5b8858f4eed4d738f8) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:12.053120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1703fb-cd80-40e0-b0ec-57368701b615 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:12.056111 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1703fb-cd80-40e0-b0ec-57368701b615 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:12.063539 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 229/451] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 18 08:25:11 2022] GET /identity/v3/role_assignments?group.id=640b54e919004efebb102e0e48da8ceb&scope.project.id=21519c13797a4d5b8858f4eed4d738f8&role.id=d2ba35e60f974010a5e7dd5c4bebecf8 => generated 611 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:25:12.064918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d36513-8c27-4e10-8955-89ececc53685 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:12.066470 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d36513-8c27-4e10-8955-89ececc53685 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:12.237720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 223/452] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:25:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:25:12.245852 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1703fb-cd80-40e0-b0ec-57368701b615 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:12.381452 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1703fb-cd80-40e0-b0ec-57368701b615 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:12.410621 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:12.426761 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd48ea-ef57-4662-9faf-04a14f756f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:12.428157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd48ea-ef57-4662-9faf-04a14f756f65 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:12.428534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd48ea-ef57-4662-9faf-04a14f756f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:12.428816 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd48ea-ef57-4662-9faf-04a14f756f65 None admin] PATH_INFO: `/v3/projects/eedba945f8d24629a712fbf156a2209f` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:12.430767 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd48ea-ef57-4662-9faf-04a14f756f65 None admin] RBAC: Authorizing `identity:get_project(project_id=eedba945f8d24629a712fbf156a2209f)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:12.433279 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd48ea-ef57-4662-9faf-04a14f756f65 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:12.436268 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 230/453] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:25:12 2022] GET /identity/v3/projects/eedba945f8d24629a712fbf156a2209f => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:12.442693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d36513-8c27-4e10-8955-89ececc53685 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:12.548275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:12.569130 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d36513-8c27-4e10-8955-89ececc53685 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:12.607132 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50a257-8329-4a34-a909-d2edeb43ad88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:12.608016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50a257-8329-4a34-a909-d2edeb43ad88 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:12.608298 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50a257-8329-4a34-a909-d2edeb43ad88 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:12.608566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50a257-8329-4a34-a909-d2edeb43ad88 None admin] PATH_INFO: `/v3/groups/76f3265f09eb410fb1b3a78791fb8fef` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:12.609775 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50a257-8329-4a34-a909-d2edeb43ad88 None admin] RBAC: Authorizing `identity:get_group(group_id=76f3265f09eb410fb1b3a78791fb8fef)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:12.611220 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50a257-8329-4a34-a909-d2edeb43ad88 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:12.612879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 224/454] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Jul 18 08:25:12 2022] GET /identity/v3/groups/76f3265f09eb410fb1b3a78791fb8fef => generated 229 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:12.619922 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd48ea-ef57-4662-9faf-04a14f756f65 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:12.722051 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd48ea-ef57-4662-9faf-04a14f756f65 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:12.757369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa32a9-e044-41ce-935d-ec7babe2fd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:12.758102 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa32a9-e044-41ce-935d-ec7babe2fd88 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:12.758773 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa32a9-e044-41ce-935d-ec7babe2fd88 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:12.758773 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa32a9-e044-41ce-935d-ec7babe2fd88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:12.759448 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa32a9-e044-41ce-935d-ec7babe2fd88 None admin] RBAC: Adding query filter params (group.id=76f3265f09eb410fb1b3a78791fb8fef, role.id=467d49b8a0d3435aaba58464cc8ef85c, scope.project.id=eedba945f8d24629a712fbf156a2209f) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:12.759827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa32a9-e044-41ce-935d-ec7babe2fd88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:12.761287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa32a9-e044-41ce-935d-ec7babe2fd88 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:12.767779 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 231/455] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Jul 18 08:25:12 2022] GET /identity/v3/role_assignments?group.id=76f3265f09eb410fb1b3a78791fb8fef&scope.project.id=eedba945f8d24629a712fbf156a2209f&role.id=467d49b8a0d3435aaba58464cc8ef85c => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:15.231458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dadd76-2f55-41cc-b7a9-fefc719643bd None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:15.232311 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dadd76-2f55-41cc-b7a9-fefc719643bd None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:15.232311 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dadd76-2f55-41cc-b7a9-fefc719643bd None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:15.233889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 225/456] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:15.245767 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db3ac0-9bb5-46cd-95c5-7197fbe72972 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:15.246568 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db3ac0-9bb5-46cd-95c5-7197fbe72972 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:15.246568 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db3ac0-9bb5-46cd-95c5-7197fbe72972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:15.272856 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-76db3ac0-9bb5-46cd-95c5-7197fbe72972 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:15.274865 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-76db3ac0-9bb5-46cd-95c5-7197fbe72972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:15.342888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:15.368898 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 232/457] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:15.395608 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dadd76-2f55-41cc-b7a9-fefc719643bd None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:15.572908 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dadd76-2f55-41cc-b7a9-fefc719643bd None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:15.630044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd2ad6-2bb2-43e9-bdf4-5217605521e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:15.630950 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd2ad6-2bb2-43e9-bdf4-5217605521e8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:15.631238 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd2ad6-2bb2-43e9-bdf4-5217605521e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:15.631726 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd2ad6-2bb2-43e9-bdf4-5217605521e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:15.640197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088237e-dea5-42e3-862a-cc24ad09e032 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:15.640731 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088237e-dea5-42e3-862a-cc24ad09e032 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:15.641347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088237e-dea5-42e3-862a-cc24ad09e032 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:15.643277 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 233/458] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:15.649332 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb20b98-0319-4cd3-aeef-28e489f80ca3 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:15.649668 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb20b98-0319-4cd3-aeef-28e489f80ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:15.649897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb20b98-0319-4cd3-aeef-28e489f80ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:15.654314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd2ad6-2bb2-43e9-bdf4-5217605521e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:15.657280 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd2ad6-2bb2-43e9-bdf4-5217605521e8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:15.687863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-fcb20b98-0319-4cd3-aeef-28e489f80ca3 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:15.689122 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fcb20b98-0319-4cd3-aeef-28e489f80ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:15.692750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-e9bd2ad6-2bb2-43e9-bdf4-5217605521e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:15.692750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:15.692750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:15.692750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:15.692750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:15.692750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:15.692750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:15.692750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:15.692750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:15.692750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:15.692750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:15.692750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:15.692750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:15.692750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:15.692750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:15.694476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:15.694476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:15.694476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:15.694476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:15.694476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:15.694476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:15.694476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:15.694476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:15.694476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:15.694476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:15.694476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:15.694476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:15.694476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:15.694476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:15.694476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:15.694476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:15.694476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:15.696419 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:15.696419 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:15.696419 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:15.696419 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:15.699621 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 226/459] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:15.707816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd2ad6-2bb2-43e9-bdf4-5217605521e8 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:15.722625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:15.808701 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 234/460] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:25:15.810521 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95d2d3-278e-4bfc-a3a0-caca5c5d8abf None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:15.811012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95d2d3-278e-4bfc-a3a0-caca5c5d8abf None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:15.811377 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95d2d3-278e-4bfc-a3a0-caca5c5d8abf None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:15.817319 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 235/461] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:15.818833 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3d0af-e574-45a0-950c-5ec8842cb4ea None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:15.819204 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3d0af-e574-45a0-950c-5ec8842cb4ea None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:15.819564 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3d0af-e574-45a0-950c-5ec8842cb4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:15.853756 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-88d3d0af-e574-45a0-950c-5ec8842cb4ea None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:15.855232 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-88d3d0af-e574-45a0-950c-5ec8842cb4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:15.869366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd2ad6-2bb2-43e9-bdf4-5217605521e8 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:15.914016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f869b-8bab-4a7f-a763-9950774a2f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:15.915098 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f869b-8bab-4a7f-a763-9950774a2f13 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:15.916264 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f869b-8bab-4a7f-a763-9950774a2f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:15.916766 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f869b-8bab-4a7f-a763-9950774a2f13 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:15.918010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f869b-8bab-4a7f-a763-9950774a2f13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:15.918781 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f869b-8bab-4a7f-a763-9950774a2f13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:15.921192 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f869b-8bab-4a7f-a763-9950774a2f13 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:15.938748 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 236/463] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:25:15.940024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3d0af-e574-45a0-950c-5ec8842cb4ea None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:15.941340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 227/463] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:15.942709 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f869b-8bab-4a7f-a763-9950774a2f13 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:16.017745 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:16.020669 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:16.059575 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f869b-8bab-4a7f-a763-9950774a2f13 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:16.079802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3d0af-e574-45a0-950c-5ec8842cb4ea None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:16.097225 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517ef43-a73b-4f24-9ffe-ce57362d8faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:16.097824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517ef43-a73b-4f24-9ffe-ce57362d8faa None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:16.098107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517ef43-a73b-4f24-9ffe-ce57362d8faa None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:16.098381 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517ef43-a73b-4f24-9ffe-ce57362d8faa None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:16.099101 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517ef43-a73b-4f24-9ffe-ce57362d8faa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:16.100581 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517ef43-a73b-4f24-9ffe-ce57362d8faa None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:16.122106 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5776fb-b179-4508-9b43-2a44a498b1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:16.123146 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5776fb-b179-4508-9b43-2a44a498b1db None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:16.123519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5776fb-b179-4508-9b43-2a44a498b1db None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:16.123942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5776fb-b179-4508-9b43-2a44a498b1db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:16.136569 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5776fb-b179-4508-9b43-2a44a498b1db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:16.138200 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5776fb-b179-4508-9b43-2a44a498b1db None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:16.159150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-0f5776fb-b179-4508-9b43-2a44a498b1db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:16.159150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:16.159150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:16.159150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:16.159150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:16.159150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:16.159150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:16.159150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:16.159150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:16.159150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:16.159150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:16.159150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:16.159150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:16.159150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:16.159150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:16.160166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:16.160166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:16.160166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:16.160166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:16.160166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:16.160166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:16.160166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:16.160166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:16.160166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:16.160166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:16.160166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:16.160166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:16.160166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:16.160166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:16.160166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:16.160166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:16.160166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:16.161006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:16.161006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:16.161006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:16.161006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:16.161874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 237/464] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:16.163018 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5776fb-b179-4508-9b43-2a44a498b1db None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:16.212168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 228/465] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 18 08:25:15 2022] POST /identity/v3/users => generated 246 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:25:16.213744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517ef43-a73b-4f24-9ffe-ce57362d8faa None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:16.306224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:16.311538 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5776fb-b179-4508-9b43-2a44a498b1db None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:16.363763 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaeb4c6-0c03-4e38-a7bc-01c2daf58579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:16.365932 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaeb4c6-0c03-4e38-a7bc-01c2daf58579 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:16.368661 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaeb4c6-0c03-4e38-a7bc-01c2daf58579 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:16.368941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaeb4c6-0c03-4e38-a7bc-01c2daf58579 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:16.388858 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaeb4c6-0c03-4e38-a7bc-01c2daf58579 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:16.391514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaeb4c6-0c03-4e38-a7bc-01c2daf58579 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:16.418330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-afaeb4c6-0c03-4e38-a7bc-01c2daf58579 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:16.418330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:16.418330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:16.418330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:16.418330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:16.418330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:16.418330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:16.418330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:16.418330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:16.418330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:16.418330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:16.418330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:16.418330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:16.418330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:16.418330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:16.419258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:16.419258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:16.419258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:16.419258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:16.419258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:16.419258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:16.419258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:16.419258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:16.419258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:16.419258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:16.419258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:16.419258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:16.419258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:16.419258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:16.419258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:16.419258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:16.419258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:16.420158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:16.420158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:16.420158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:16.420158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:16.423701 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 238/466] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:16.423701 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95716f85-54c9-450a-953a-87a73b14c80a None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:16.423701 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95716f85-54c9-450a-953a-87a73b14c80a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:16.423701 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95716f85-54c9-450a-953a-87a73b14c80a None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:16.427784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 239/467] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:16.428804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95716f85-54c9-450a-953a-87a73b14c80a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:16.440723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517ef43-a73b-4f24-9ffe-ce57362d8faa None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:16.473904 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:16.487215 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04adf2f-ef31-4dee-ba46-800ff10c60d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:16.498694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04adf2f-ef31-4dee-ba46-800ff10c60d1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:16.498694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04adf2f-ef31-4dee-ba46-800ff10c60d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:16.498694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04adf2f-ef31-4dee-ba46-800ff10c60d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:16.498694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04adf2f-ef31-4dee-ba46-800ff10c60d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:16.498694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04adf2f-ef31-4dee-ba46-800ff10c60d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:16.498694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04adf2f-ef31-4dee-ba46-800ff10c60d1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:16.505815 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 229/468] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:16.507108 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d17c3-749f-4dda-8480-218c91a52288 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:16.507396 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d17c3-749f-4dda-8480-218c91a52288 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:16.508946 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d17c3-749f-4dda-8480-218c91a52288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:16.529877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-891d17c3-749f-4dda-8480-218c91a52288 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:16.531083 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-891d17c3-749f-4dda-8480-218c91a52288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:16.582818 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95716f85-54c9-450a-953a-87a73b14c80a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:16.623371 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 230/469] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:16.624158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d17c3-749f-4dda-8480-218c91a52288 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:16.638501 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621dd3e4-dbeb-4c53-abaa-ab147d39555d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:16.639765 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dd3e4-dbeb-4c53-abaa-ab147d39555d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:16.640203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dd3e4-dbeb-4c53-abaa-ab147d39555d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:16.640634 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dd3e4-dbeb-4c53-abaa-ab147d39555d None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:16.642476 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621dd3e4-dbeb-4c53-abaa-ab147d39555d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:16.642972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621dd3e4-dbeb-4c53-abaa-ab147d39555d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:16.645202 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621dd3e4-dbeb-4c53-abaa-ab147d39555d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:16.670324 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 240/470] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:16.671310 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621dd3e4-dbeb-4c53-abaa-ab147d39555d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:16.687375 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:16.769624 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d17c3-749f-4dda-8480-218c91a52288 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:16.790069 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:16.797506 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621dd3e4-dbeb-4c53-abaa-ab147d39555d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:16.821603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b09570-ba93-4646-90b2-76ed93a72228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:16.823176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b09570-ba93-4646-90b2-76ed93a72228 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:16.823643 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b09570-ba93-4646-90b2-76ed93a72228 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:16.824368 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b09570-ba93-4646-90b2-76ed93a72228 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:16.825234 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b09570-ba93-4646-90b2-76ed93a72228 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:16.827831 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b09570-ba93-4646-90b2-76ed93a72228 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:16.849904 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33ce9d-a6ba-43ef-ac9e-f56d4324a31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:16.850779 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33ce9d-a6ba-43ef-ac9e-f56d4324a31b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:16.851115 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33ce9d-a6ba-43ef-ac9e-f56d4324a31b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:16.852061 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33ce9d-a6ba-43ef-ac9e-f56d4324a31b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:16.866199 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33ce9d-a6ba-43ef-ac9e-f56d4324a31b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:16.868165 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33ce9d-a6ba-43ef-ac9e-f56d4324a31b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:16.891795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-7e33ce9d-a6ba-43ef-ac9e-f56d4324a31b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:16.891795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:16.891795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:16.891795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:16.891795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:16.891795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:16.891795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:16.891795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:16.891795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:16.891795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:16.891795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:16.891795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:16.891795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:16.891795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:16.891795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:16.892671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:16.892671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:16.892671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:16.892671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:16.892671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:16.892671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:16.892671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:16.892671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:16.892671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:16.892671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:16.892671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:16.892671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:16.892671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:16.892671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:16.892671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:16.892671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:16.892671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:16.893567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:16.893567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:16.893567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:16.893567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:16.893567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 241/471] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:16.893567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33ce9d-a6ba-43ef-ac9e-f56d4324a31b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:16.911635 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 231/472] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 18 08:25:16 2022] POST /identity/v3/users => generated 248 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:25:16.916760 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b09570-ba93-4646-90b2-76ed93a72228 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:17.018348 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:17.079778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33ce9d-a6ba-43ef-ac9e-f56d4324a31b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:17.123194 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20b82f-6e33-4d4a-aa77-f436c53c69ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:17.123914 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20b82f-6e33-4d4a-aa77-f436c53c69ea None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:17.124203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20b82f-6e33-4d4a-aa77-f436c53c69ea None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:17.124483 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20b82f-6e33-4d4a-aa77-f436c53c69ea None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:17.125209 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd20b82f-6e33-4d4a-aa77-f436c53c69ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:17.126666 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd20b82f-6e33-4d4a-aa77-f436c53c69ea None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:17.134497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b09570-ba93-4646-90b2-76ed93a72228 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:17.189094 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba7c8d5-46c3-403a-8aa4-9b73c01230d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:17.189860 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7c8d5-46c3-403a-8aa4-9b73c01230d4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:17.190151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7c8d5-46c3-403a-8aa4-9b73c01230d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:17.190439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7c8d5-46c3-403a-8aa4-9b73c01230d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:17.191197 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba7c8d5-46c3-403a-8aa4-9b73c01230d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:17.191514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba7c8d5-46c3-403a-8aa4-9b73c01230d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:17.192941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba7c8d5-46c3-403a-8aa4-9b73c01230d4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:17.203955 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 242/473] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 18 08:25:16 2022] POST /identity/v3/users => generated 248 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:25:17.206292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679951fc-f728-4693-9e27-1c94846f4df6 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:17.206587 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679951fc-f728-4693-9e27-1c94846f4df6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:17.206867 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679951fc-f728-4693-9e27-1c94846f4df6 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:17.208467 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 243/474] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:17.211338 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 232/475] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:17.218650 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679951fc-f728-4693-9e27-1c94846f4df6 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:17.222223 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f682811d-1a40-4f3d-8d3f-c1d21d67f5e3 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:17.223757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f682811d-1a40-4f3d-8d3f-c1d21d67f5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:17.224054 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f682811d-1a40-4f3d-8d3f-c1d21d67f5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:17.249131 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-f682811d-1a40-4f3d-8d3f-c1d21d67f5e3 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:17.250877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f682811d-1a40-4f3d-8d3f-c1d21d67f5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:17.288420 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:17.398518 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 233/476] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:17.410298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679951fc-f728-4693-9e27-1c94846f4df6 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:17.422522 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f682811d-1a40-4f3d-8d3f-c1d21d67f5e3 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:17.434080 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:17.462207 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24323bca-b87c-43fa-8dc4-9da0ef0f2db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:17.463127 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24323bca-b87c-43fa-8dc4-9da0ef0f2db4 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:17.464433 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24323bca-b87c-43fa-8dc4-9da0ef0f2db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:17.464719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24323bca-b87c-43fa-8dc4-9da0ef0f2db4 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:17.465447 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24323bca-b87c-43fa-8dc4-9da0ef0f2db4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:17.466959 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24323bca-b87c-43fa-8dc4-9da0ef0f2db4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:17.593544 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f682811d-1a40-4f3d-8d3f-c1d21d67f5e3 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:17.601037 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 244/477] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 18 08:25:17 2022] POST /identity/v3/users => generated 249 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:25:17.643550 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f9787-d688-417c-ac91-4b12df987c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:17.644294 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f9787-d688-417c-ac91-4b12df987c12 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:17.644634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f9787-d688-417c-ac91-4b12df987c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:17.644905 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f9787-d688-417c-ac91-4b12df987c12 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:17.645808 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f9787-d688-417c-ac91-4b12df987c12 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:17.647227 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f9787-d688-417c-ac91-4b12df987c12 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:17.686613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.sql.core [None req-296f9787-d688-417c-ac91-4b12df987c12 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 08:25:17.686613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 08:25:17.686613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [parameters: {'id': 'd30304262f444e799a441bda46512035', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 08:25:17.686613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80338) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 08:25:17.711540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-296f9787-d688-417c-ac91-4b12df987c12 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 08:25:17.711540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:17.711540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 08:25:17.711540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 08:25:17.711540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 08:25:17.711540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 08:25:17.711540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 08:25:17.711540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 08:25:17.711540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 08:25:17.711540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application conn.query(q) Jul 18 08:25:17.711540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 08:25:17.711540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 08:25:17.711540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 08:25:17.711540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application result.read() Jul 18 08:25:17.711540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 08:25:17.712813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 08:25:17.712813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 08:25:17.712813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 08:25:17.712813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 08:25:17.712813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 08:25:17.712813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 08:25:17.712813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 08:25:17.712813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 08:25:17.712813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:17.712813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 08:25:17.712813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:17.712813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:17.712813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 08:25:17.712813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 08:25:17.712813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 18 08:25:17.712813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return ref.to_dict() Jul 18 08:25:17.712813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 08:25:17.712813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application next(self.gen) Jul 18 08:25:17.713803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 08:25:17.713803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application yield resource Jul 18 08:25:17.713803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 08:25:17.713803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application next(self.gen) Jul 18 08:25:17.713803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 08:25:17.713803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 08:25:17.713803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 08:25:17.713803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 08:25:17.713803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 08:25:17.713803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise self.value Jul 18 08:25:17.713803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 08:25:17.713803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 08:25:17.713803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 08:25:17.713803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application session.commit() Jul 18 08:25:17.713803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 08:25:17.713803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 08:25:17.713803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 08:25:17.715191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 08:25:17.715191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 08:25:17.715191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self.session.flush() Jul 18 08:25:17.715191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 08:25:17.715191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 08:25:17.715191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 08:25:17.715191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 08:25:17.715191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 08:25:17.715191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application compat.raise_( Jul 18 08:25:17.715191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 08:25:17.715191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception Jul 18 08:25:17.715191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 08:25:17.715191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 08:25:17.715191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 08:25:17.715191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 08:25:17.715191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 08:25:17.715191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 08:25:17.716201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 08:25:17.716201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 08:25:17.716201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 08:25:17.716201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 08:25:17.716201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 08:25:17.716201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 08:25:17.716201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 08:25:17.716201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 08:25:17.716201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 08:25:17.716201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 08:25:17.716201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 08:25:17.716201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 08:25:17.716201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 08:25:17.716201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 08:25:17.716201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 08:25:17.716201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception Jul 18 08:25:17.717037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 08:25:17.717037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 08:25:17.717037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 08:25:17.717037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 08:25:17.717037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 08:25:17.717037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 08:25:17.717037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 08:25:17.717037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application conn.query(q) Jul 18 08:25:17.717037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 08:25:17.717037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 08:25:17.717037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 08:25:17.717037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application result.read() Jul 18 08:25:17.717037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 08:25:17.717037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 08:25:17.717037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 08:25:17.717037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 08:25:17.717847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 08:25:17.717847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 08:25:17.717847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 08:25:17.717847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 08:25:17.717847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 08:25:17.717847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 08:25:17.717847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [parameters: {'id': 'd30304262f444e799a441bda46512035', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 08:25:17.717847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 08:25:17.717847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:17.717847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:25:17.717847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:17.717847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:17.717847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:17.717847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:17.717847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:17.717847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:17.718658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:17.718658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:17.718658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:17.718658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:17.718658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:17.718658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:17.718658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 18 08:25:17.718658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 18 08:25:17.718658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:17.718658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:17.718658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 18 08:25:17.718658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 18 08:25:17.718658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 08:25:17.718658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 08:25:17.718658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 08:25:17.718658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:17.719523 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 234/478] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Jul 18 08:25:17 2022] POST /identity/v3/roles => generated 144 bytes in 291 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:17.724145 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24323bca-b87c-43fa-8dc4-9da0ef0f2db4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:17.933077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24323bca-b87c-43fa-8dc4-9da0ef0f2db4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:17.969550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3f66c-f706-43be-bc9a-25def9b52bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:17.970294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3f66c-f706-43be-bc9a-25def9b52bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:17.970577 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3f66c-f706-43be-bc9a-25def9b52bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:17.970854 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3f66c-f706-43be-bc9a-25def9b52bd9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:17.982124 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb3f66c-f706-43be-bc9a-25def9b52bd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:17.990495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-dfb3f66c-f706-43be-bc9a-25def9b52bd9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:25:17.990495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:17.990495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:17.990495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:17.990495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:17.990495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:17.990495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:17.990495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:17.990495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:17.990495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:17.990495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:17.990495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:17.990495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:17.990495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:17.990495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:17.990495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:17.991417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:17.991417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:17.991417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:17.991417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:17.991417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:17.991417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:17.991417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:17.991417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:17.991417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:17.991417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:17.991417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:17.991417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:17.991417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:17.991417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:17.991417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:17.991417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:17.992823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:17.992823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:17.992823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:17.992823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:17.992823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:17.992823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:17.992823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:17.992823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:17.992823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:17.992823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:17.992823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 08:25:17.992823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:17.992823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 245/479] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:25:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:18.001930 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f9787-d688-417c-ac91-4b12df987c12 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:18.055769 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:18.206002 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f9787-d688-417c-ac91-4b12df987c12 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:18.276194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c6bc9-bb35-4160-b604-416bf5b68f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:18.277264 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c6bc9-bb35-4160-b604-416bf5b68f29 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:18.277772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c6bc9-bb35-4160-b604-416bf5b68f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:18.278261 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c6bc9-bb35-4160-b604-416bf5b68f29 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:18.279299 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c6bc9-bb35-4160-b604-416bf5b68f29 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:18.279866 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c6bc9-bb35-4160-b604-416bf5b68f29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:18.282527 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c6bc9-bb35-4160-b604-416bf5b68f29 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:18.297638 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 235/480] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:19.093229 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e8dd0-e960-4f37-84ed-33e727542960 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:19.093734 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e8dd0-e960-4f37-84ed-33e727542960 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:19.093734 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e8dd0-e960-4f37-84ed-33e727542960 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:19.095401 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 246/481] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:19.102667 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b3767-4242-41b5-b17f-ccfe74e70da9 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:19.103271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b3767-4242-41b5-b17f-ccfe74e70da9 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:19.103271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b3767-4242-41b5-b17f-ccfe74e70da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:19.127190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-ce6b3767-4242-41b5-b17f-ccfe74e70da9 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:19.128442 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ce6b3767-4242-41b5-b17f-ccfe74e70da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:19.217590 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 236/482] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:25:19.246975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e8dd0-e960-4f37-84ed-33e727542960 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:19.311381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:19.423837 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e8dd0-e960-4f37-84ed-33e727542960 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:19.484888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a55a4-8b79-4808-b836-36b414aacc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:19.486704 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a55a4-8b79-4808-b836-36b414aacc25 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:19.487526 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a55a4-8b79-4808-b836-36b414aacc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:19.488312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a55a4-8b79-4808-b836-36b414aacc25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:19.512305 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a55a4-8b79-4808-b836-36b414aacc25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:19.522224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-bd5a55a4-8b79-4808-b836-36b414aacc25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:19.522224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:19.522224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:19.522224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:19.522224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:19.522224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:19.522224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:19.522224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:19.522224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:19.522224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:19.522224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:19.522224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:19.522224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:19.522224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:19.522224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:19.522224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:19.523945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:19.523945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:19.523945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:19.523945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:19.523945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:19.523945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:19.523945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:19.523945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:19.523945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:19.523945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:19.523945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:19.523945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:19.523945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:19.523945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:19.523945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:19.523945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:19.525233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:19.525233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:19.525233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:19.525233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:19.525233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:19.525233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:19.525233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:19.525233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:19.525233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:19.525233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:19.525233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:19.525233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:19.533800 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 247/483] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:25:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:19.543555 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b3767-4242-41b5-b17f-ccfe74e70da9 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:19.584724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:19.737182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b3767-4242-41b5-b17f-ccfe74e70da9 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:19.797186 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b72d0-85cd-46ed-ac71-6f2e7e238501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:19.798376 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b72d0-85cd-46ed-ac71-6f2e7e238501 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:19.798862 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b72d0-85cd-46ed-ac71-6f2e7e238501 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:19.799331 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b72d0-85cd-46ed-ac71-6f2e7e238501 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:19.800355 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852b72d0-85cd-46ed-ac71-6f2e7e238501 None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:19.800879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852b72d0-85cd-46ed-ac71-6f2e7e238501 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:19.802969 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852b72d0-85cd-46ed-ac71-6f2e7e238501 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:19.804033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc4659-5cdd-4a5f-9a74-92e60f7c8178 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:19.805081 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc4659-5cdd-4a5f-9a74-92e60f7c8178 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:19.805377 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc4659-5cdd-4a5f-9a74-92e60f7c8178 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:19.807253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 248/484] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:25:19.814023 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccc517-d094-4a9a-8d61-3a468d697331 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:19.814331 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccc517-d094-4a9a-8d61-3a468d697331 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:19.814594 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccc517-d094-4a9a-8d61-3a468d697331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:19.826639 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 237/485] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:25:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:19.848244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b72d0-85cd-46ed-ac71-6f2e7e238501 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:19.850156 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-56ccc517-d094-4a9a-8d61-3a468d697331 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:19.853159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-56ccc517-d094-4a9a-8d61-3a468d697331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:20.004332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:20.011883 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 249/486] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:25:20.013560 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0fc08-dac6-4702-a6c6-865d3dd83f88 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:20.013854 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0fc08-dac6-4702-a6c6-865d3dd83f88 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:20.014301 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0fc08-dac6-4702-a6c6-865d3dd83f88 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:20.016008 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 250/487] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:20.022002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f4500-c552-486a-856c-87e8df5046e3 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:20.022562 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f4500-c552-486a-856c-87e8df5046e3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:20.022850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f4500-c552-486a-856c-87e8df5046e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:20.041085 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b72d0-85cd-46ed-ac71-6f2e7e238501 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:20.042371 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-fb3f4500-c552-486a-856c-87e8df5046e3 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:20.043720 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fb3f4500-c552-486a-856c-87e8df5046e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:20.065036 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:20.105076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68de42-b6df-4a2a-bd33-90814715169c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:20.107334 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68de42-b6df-4a2a-bd33-90814715169c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:20.107967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68de42-b6df-4a2a-bd33-90814715169c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:20.108446 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68de42-b6df-4a2a-bd33-90814715169c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:20.131734 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68de42-b6df-4a2a-bd33-90814715169c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:20.134729 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68de42-b6df-4a2a-bd33-90814715169c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:20.135830 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 251/488] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:25:20.136878 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f4500-c552-486a-856c-87e8df5046e3 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:20.163579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-5f68de42-b6df-4a2a-bd33-90814715169c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:20.163579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:20.163579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:20.163579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:20.163579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:20.163579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:20.163579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:20.163579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:20.163579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:20.163579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:20.163579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:20.163579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:20.163579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:20.163579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:20.163579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:20.165078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:20.165078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:20.165078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:20.165078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:20.165078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:20.165078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:20.165078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:20.165078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:20.165078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:20.165078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:20.165078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:20.165078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:20.165078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:20.165078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:20.165078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:20.165078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:20.165078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:20.165930 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:20.165930 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:20.165930 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:20.165930 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:20.168957 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 238/489] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:20.170809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68de42-b6df-4a2a-bd33-90814715169c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:20.323200 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f4500-c552-486a-856c-87e8df5046e3 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:20.346105 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68de42-b6df-4a2a-bd33-90814715169c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:20.368962 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6e85b-0165-443f-b122-5e320eeb8a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:20.369790 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6e85b-0165-443f-b122-5e320eeb8a25 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:20.370078 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6e85b-0165-443f-b122-5e320eeb8a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:20.370363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6e85b-0165-443f-b122-5e320eeb8a25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:20.384239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6e85b-0165-443f-b122-5e320eeb8a25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:20.392907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-c0c6e85b-0165-443f-b122-5e320eeb8a25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:20.392907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:20.392907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:20.392907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:20.392907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:20.392907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:20.392907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:20.392907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:20.392907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:20.392907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:20.392907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:20.392907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:20.392907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:20.392907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:20.392907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:20.392907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:20.393890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:20.393890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:20.393890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:20.393890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:20.393890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:20.393890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:20.393890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:20.393890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:20.393890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:20.393890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:20.393890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:20.393890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:20.393890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:20.393890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:20.393890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:20.393890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:20.394965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:20.394965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:20.394965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:20.394965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:20.394965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:20.394965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:20.394965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:20.394965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:20.394965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:20.394965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:20.394965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:20.394965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:20.394965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 252/490] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:25:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:20.396113 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6e85b-0165-443f-b122-5e320eeb8a25 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:20.412831 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:20.426208 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636739d-6fa6-46ae-8679-90bfdc90b142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:20.426635 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:20.428264 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636739d-6fa6-46ae-8679-90bfdc90b142 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:20.428902 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636739d-6fa6-46ae-8679-90bfdc90b142 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:20.430009 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636739d-6fa6-46ae-8679-90bfdc90b142 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:20.452116 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636739d-6fa6-46ae-8679-90bfdc90b142 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:20.468184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-b636739d-6fa6-46ae-8679-90bfdc90b142 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:20.468184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:20.468184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:20.468184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:20.468184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:20.468184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:20.468184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:20.468184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:20.468184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:20.468184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:20.468184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:20.468184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:20.468184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:20.468184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:20.468184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:20.468184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:20.469230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:20.469230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:20.469230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:20.469230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:20.469230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:20.469230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:20.469230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:20.469230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:20.469230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:20.469230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:20.469230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:20.469230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:20.469230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:20.469230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:20.469230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:20.469230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:20.470176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:20.470176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:20.470176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:20.470176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:20.470176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:20.470176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:20.470176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:20.470176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:20.470176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:20.470176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:20.470176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:20.470176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:20.480051 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 239/491] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:25:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:20.481853 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636739d-6fa6-46ae-8679-90bfdc90b142 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:20.564186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6e85b-0165-443f-b122-5e320eeb8a25 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:20.613648 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1afe6-84e0-4dca-a8b7-dd2d76db811e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:20.614792 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1afe6-84e0-4dca-a8b7-dd2d76db811e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:20.615478 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1afe6-84e0-4dca-a8b7-dd2d76db811e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:20.615917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1afe6-84e0-4dca-a8b7-dd2d76db811e None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:20.617046 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1afe6-84e0-4dca-a8b7-dd2d76db811e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:20.617729 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1afe6-84e0-4dca-a8b7-dd2d76db811e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:20.619361 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1afe6-84e0-4dca-a8b7-dd2d76db811e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:20.640390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 253/492] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:20.642230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1afe6-84e0-4dca-a8b7-dd2d76db811e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:20.692683 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636739d-6fa6-46ae-8679-90bfdc90b142 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:20.763693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f16376-2f6b-4e9e-bddb-44ea4e9984dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:20.764909 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f16376-2f6b-4e9e-bddb-44ea4e9984dd None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:20.765411 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f16376-2f6b-4e9e-bddb-44ea4e9984dd None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:20.765914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f16376-2f6b-4e9e-bddb-44ea4e9984dd None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:20.767575 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f16376-2f6b-4e9e-bddb-44ea4e9984dd None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:20.768194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f16376-2f6b-4e9e-bddb-44ea4e9984dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:20.770928 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f16376-2f6b-4e9e-bddb-44ea4e9984dd None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:20.789400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 240/493] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:25:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:20.790677 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f16376-2f6b-4e9e-bddb-44ea4e9984dd None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:20.805695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:20.843807 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1afe6-84e0-4dca-a8b7-dd2d76db811e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:20.888800 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30b4f4-b056-442f-89fd-28d4a257dda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:20.889401 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30b4f4-b056-442f-89fd-28d4a257dda7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:20.890365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30b4f4-b056-442f-89fd-28d4a257dda7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:20.890365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30b4f4-b056-442f-89fd-28d4a257dda7 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:20.890811 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30b4f4-b056-442f-89fd-28d4a257dda7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:20.891054 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30b4f4-b056-442f-89fd-28d4a257dda7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:20.892569 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30b4f4-b056-442f-89fd-28d4a257dda7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:20.899203 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:20.910334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 254/494] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:25:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:20.910334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b236843-c92c-4299-b319-280885d8c326 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:20.910334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b236843-c92c-4299-b319-280885d8c326 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:20.910334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b236843-c92c-4299-b319-280885d8c326 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:20.910334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 255/495] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:20.910334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b236843-c92c-4299-b319-280885d8c326 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:21.000519 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f16376-2f6b-4e9e-bddb-44ea4e9984dd None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:21.055390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7629a89-7c78-4dfb-907c-ff40a7d9da44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:21.056337 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7629a89-7c78-4dfb-907c-ff40a7d9da44 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:21.056625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7629a89-7c78-4dfb-907c-ff40a7d9da44 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:21.056902 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7629a89-7c78-4dfb-907c-ff40a7d9da44 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:21.065844 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7629a89-7c78-4dfb-907c-ff40a7d9da44 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:21.067407 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7629a89-7c78-4dfb-907c-ff40a7d9da44 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:21.086269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-e7629a89-7c78-4dfb-907c-ff40a7d9da44 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 08:25:21.086269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:21.086269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:21.086269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:21.086269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:21.086269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:21.086269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:21.086269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:21.086269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:21.086269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:21.086269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:21.086269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:21.086269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:25:21.086269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:25:21.086269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:25:21.086269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:25:21.089282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:21.089282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:21.089282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:25:21.089282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:21.089282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:25:21.089282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:21.089282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:21.089282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:21.089282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:21.089282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:21.089282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:21.089282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:21.089282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:21.089282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:21.089282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:21.089282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:21.089282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:21.089282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:21.091150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:21.091150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:21.091150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:25:21.091150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:25:21.091150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:25:21.091150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:25:21.091150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:25:21.091150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:25:21.091150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 08:25:21.091150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:21.091150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 241/496] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 18 08:25:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:21.091150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7629a89-7c78-4dfb-907c-ff40a7d9da44 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:21.119401 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b236843-c92c-4299-b319-280885d8c326 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:21.178456 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345efa7e-06fb-4ba7-a179-e8e7f756f60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:21.179400 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345efa7e-06fb-4ba7-a179-e8e7f756f60a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:21.179910 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345efa7e-06fb-4ba7-a179-e8e7f756f60a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:21.180373 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345efa7e-06fb-4ba7-a179-e8e7f756f60a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:21.185726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:21.200538 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345efa7e-06fb-4ba7-a179-e8e7f756f60a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:21.207238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345efa7e-06fb-4ba7-a179-e8e7f756f60a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:21.229253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-345efa7e-06fb-4ba7-a179-e8e7f756f60a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:21.229253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:21.229253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:21.229253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:21.229253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:21.229253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:21.229253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:21.229253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:21.229253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:21.229253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:21.229253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:21.229253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:21.229253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:21.229253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:21.229253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:21.230683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:21.230683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:21.230683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:21.230683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:21.230683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:21.230683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:21.230683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:21.230683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:21.230683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:21.230683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:21.230683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:21.230683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:21.230683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:21.230683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:21.230683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:21.230683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:21.230683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:21.231764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:21.231764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:21.231764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:21.231764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:21.234231 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 256/497] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:21.235899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f53fb2-b973-4977-b753-1955ce915427 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:21.236366 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f53fb2-b973-4977-b753-1955ce915427 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:21.236840 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f53fb2-b973-4977-b753-1955ce915427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:21.257354 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7629a89-7c78-4dfb-907c-ff40a7d9da44 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:21.258542 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e2f53fb2-b973-4977-b753-1955ce915427 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:21.260202 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e2f53fb2-b973-4977-b753-1955ce915427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:21.279768 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:21.316132 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b505a7-cc3b-47d0-b787-6533db17ce88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:21.317546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b505a7-cc3b-47d0-b787-6533db17ce88 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:21.318165 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b505a7-cc3b-47d0-b787-6533db17ce88 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:21.318660 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b505a7-cc3b-47d0-b787-6533db17ce88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:21.342341 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b505a7-cc3b-47d0-b787-6533db17ce88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:21.345173 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b505a7-cc3b-47d0-b787-6533db17ce88 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:21.364381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 257/498] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:21.365801 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f53fb2-b973-4977-b753-1955ce915427 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:21.380648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-29b505a7-cc3b-47d0-b787-6533db17ce88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:21.380648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:21.380648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:21.380648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:21.380648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:21.380648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:21.380648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:21.380648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:21.380648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:21.380648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:21.380648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:21.380648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:21.380648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:21.380648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:21.380648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:21.382398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:21.382398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:21.382398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:21.382398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:21.382398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:21.382398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:21.382398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:21.382398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:21.382398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:21.382398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:21.382398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:21.382398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:21.382398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:21.382398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:21.382398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:21.382398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:21.382398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:21.384625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:21.384625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:21.384625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:21.384625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:21.391061 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 242/499] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:21.392769 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b505a7-cc3b-47d0-b787-6533db17ce88 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:21.513803 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f53fb2-b973-4977-b753-1955ce915427 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:21.534903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:21.555242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20505c0a-4140-4d73-bb8c-565cce6279e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:21.556056 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20505c0a-4140-4d73-bb8c-565cce6279e1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:21.556338 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20505c0a-4140-4d73-bb8c-565cce6279e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:21.556608 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20505c0a-4140-4d73-bb8c-565cce6279e1 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:21.557588 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20505c0a-4140-4d73-bb8c-565cce6279e1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:21.557804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20505c0a-4140-4d73-bb8c-565cce6279e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:21.559319 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20505c0a-4140-4d73-bb8c-565cce6279e1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:21.564343 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b505a7-cc3b-47d0-b787-6533db17ce88 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:21.600333 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 258/500] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 18 08:25:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:21.601023 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20505c0a-4140-4d73-bb8c-565cce6279e1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:21.622783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6753d-85e6-4d74-8f4a-8c756ee94332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:21.623479 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6753d-85e6-4d74-8f4a-8c756ee94332 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:21.623801 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6753d-85e6-4d74-8f4a-8c756ee94332 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:21.624086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6753d-85e6-4d74-8f4a-8c756ee94332 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:21.624970 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6753d-85e6-4d74-8f4a-8c756ee94332 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:21.625282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6753d-85e6-4d74-8f4a-8c756ee94332 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:21.626739 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6753d-85e6-4d74-8f4a-8c756ee94332 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:21.642142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 243/501] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:21.643595 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6753d-85e6-4d74-8f4a-8c756ee94332 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:21.678263 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:21.732278 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20505c0a-4140-4d73-bb8c-565cce6279e1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:21.777575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560132d5-94d7-457c-a13a-33d5cf96fcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:21.778576 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560132d5-94d7-457c-a13a-33d5cf96fcb7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:21.779011 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560132d5-94d7-457c-a13a-33d5cf96fcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:21.779464 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560132d5-94d7-457c-a13a-33d5cf96fcb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:21.780771 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560132d5-94d7-457c-a13a-33d5cf96fcb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:21.781274 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560132d5-94d7-457c-a13a-33d5cf96fcb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:21.783602 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560132d5-94d7-457c-a13a-33d5cf96fcb7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:21.802191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6753d-85e6-4d74-8f4a-8c756ee94332 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:21.804248 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 259/502] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:21.805737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcceeba4-7975-48f0-812b-4e26a46d7574 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:21.806032 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcceeba4-7975-48f0-812b-4e26a46d7574 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:21.806311 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcceeba4-7975-48f0-812b-4e26a46d7574 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:21.807710 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 260/503] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:21.808529 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcceeba4-7975-48f0-812b-4e26a46d7574 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:21.863887 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa4242-b147-4464-bae0-a91db3ee8737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:21.865250 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa4242-b147-4464-bae0-a91db3ee8737 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:21.865452 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:21.865725 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa4242-b147-4464-bae0-a91db3ee8737 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:21.866187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa4242-b147-4464-bae0-a91db3ee8737 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:21.879217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa4242-b147-4464-bae0-a91db3ee8737 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:21.889103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-85fa4242-b147-4464-bae0-a91db3ee8737 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:21.889103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:21.889103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:21.889103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:21.889103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:21.889103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:21.889103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:21.889103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:21.889103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:21.889103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:21.889103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:21.889103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:21.889103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:21.889103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:21.889103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:21.889103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:21.891142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:21.891142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:21.891142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:21.891142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:21.891142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:21.891142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:21.891142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:21.891142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:21.891142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:21.891142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:21.891142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:21.891142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:21.891142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:21.891142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:21.891142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:21.891142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:21.892966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:21.892966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:21.892966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:21.892966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:21.892966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:21.892966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:21.892966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:21.892966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:21.892966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:21.892966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:21.892966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:21.892966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:21.892966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 244/504] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:25:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:21.893660 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa4242-b147-4464-bae0-a91db3ee8737 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:21.961786 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcceeba4-7975-48f0-812b-4e26a46d7574 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:22.012375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:22.014703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e81fdd-f577-4516-8701-3c511d679bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:22.015417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e81fdd-f577-4516-8701-3c511d679bac None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:22.015816 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e81fdd-f577-4516-8701-3c511d679bac None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:22.016218 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e81fdd-f577-4516-8701-3c511d679bac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:22.035175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa4242-b147-4464-bae0-a91db3ee8737 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:22.081266 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2244ecf-5094-4165-8e98-635ce308cb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:22.082523 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2244ecf-5094-4165-8e98-635ce308cb29 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:22.082936 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2244ecf-5094-4165-8e98-635ce308cb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:22.083393 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2244ecf-5094-4165-8e98-635ce308cb29 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:22.096583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2244ecf-5094-4165-8e98-635ce308cb29 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:22.099095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2244ecf-5094-4165-8e98-635ce308cb29 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:22.113805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e81fdd-f577-4516-8701-3c511d679bac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:22.116691 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e81fdd-f577-4516-8701-3c511d679bac None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:22.125510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-b2244ecf-5094-4165-8e98-635ce308cb29 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 08:25:22.125510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:22.125510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:22.125510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:22.125510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:22.125510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:22.125510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:22.125510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:22.125510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:22.125510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:22.125510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:22.125510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:22.125510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:25:22.125510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:25:22.125510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:25:22.125510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:25:22.127347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:22.127347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:22.127347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:25:22.127347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:22.127347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:25:22.127347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:22.127347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:22.127347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:22.127347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:22.127347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:22.127347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:22.127347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:22.127347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:22.127347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:22.127347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:22.127347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:22.127347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:22.127347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:22.129365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:22.129365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:22.129365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:25:22.129365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:25:22.129365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:25:22.129365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:25:22.129365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:25:22.129365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:25:22.129365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 08:25:22.129365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:22.129365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 245/505] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:25:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:22.129365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2244ecf-5094-4165-8e98-635ce308cb29 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:22.165249 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:22.249171 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2244ecf-5094-4165-8e98-635ce308cb29 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:22.258806 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 261/506] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:25:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:25:22.260823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e9c65-8695-47cf-8d28-d51f004f666a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:22.261309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e9c65-8695-47cf-8d28-d51f004f666a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:22.261793 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e9c65-8695-47cf-8d28-d51f004f666a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:22.284974 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e62e9c65-8695-47cf-8d28-d51f004f666a None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:22.286542 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e62e9c65-8695-47cf-8d28-d51f004f666a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:22.290212 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e4740-8a0d-481a-ab12-f08d192101c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:22.291535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e4740-8a0d-481a-ab12-f08d192101c8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:22.292057 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e4740-8a0d-481a-ab12-f08d192101c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:22.292556 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e4740-8a0d-481a-ab12-f08d192101c8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:22.307755 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e4740-8a0d-481a-ab12-f08d192101c8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:22.310510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e4740-8a0d-481a-ab12-f08d192101c8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:22.339270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-ef3e4740-8a0d-481a-ab12-f08d192101c8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 08:25:22.339270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:22.339270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:22.339270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:22.339270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:22.339270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:22.339270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:22.339270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:22.339270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:22.339270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:22.339270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:22.339270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:22.339270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:25:22.339270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:25:22.339270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:25:22.339270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:25:22.340585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:22.340585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:22.340585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:25:22.340585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:22.340585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:25:22.340585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:22.340585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:22.340585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:22.340585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:22.340585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:22.340585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:22.340585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:22.340585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:22.340585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:22.340585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:22.340585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:22.340585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:22.340585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:22.341988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:22.341988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:22.341988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:25:22.341988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:25:22.341988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:25:22.341988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:25:22.341988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:25:22.341988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:25:22.341988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 08:25:22.341988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:22.341988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 246/507] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:25:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:22.343068 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e4740-8a0d-481a-ab12-f08d192101c8 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:22.348525 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:22.368186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 262/508] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:22.368943 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e9c65-8695-47cf-8d28-d51f004f666a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:22.447235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:22.483018 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e4740-8a0d-481a-ab12-f08d192101c8 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:22.497825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e9c65-8695-47cf-8d28-d51f004f666a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:22.525425 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d3526-fd66-47f7-948d-3935e19268fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:22.526316 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d3526-fd66-47f7-948d-3935e19268fc None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:22.526636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d3526-fd66-47f7-948d-3935e19268fc None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:22.526949 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d3526-fd66-47f7-948d-3935e19268fc None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:22.527929 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d3526-fd66-47f7-948d-3935e19268fc None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:22.528370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d3526-fd66-47f7-948d-3935e19268fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:22.530080 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d3526-fd66-47f7-948d-3935e19268fc None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:22.541203 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 247/509] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:25:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:22.542271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d3526-fd66-47f7-948d-3935e19268fc None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:22.542831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca5539-dc85-4561-aeef-dd8923e359e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:22.544274 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca5539-dc85-4561-aeef-dd8923e359e5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:22.544771 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca5539-dc85-4561-aeef-dd8923e359e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:22.545234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca5539-dc85-4561-aeef-dd8923e359e5 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:22.547043 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca5539-dc85-4561-aeef-dd8923e359e5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:22.547834 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca5539-dc85-4561-aeef-dd8923e359e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:22.550260 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca5539-dc85-4561-aeef-dd8923e359e5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:22.584917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 263/510] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Jul 18 08:25:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:22.586749 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca5539-dc85-4561-aeef-dd8923e359e5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:22.625886 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:22.673537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d3526-fd66-47f7-948d-3935e19268fc None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:22.717605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600fd8c-13a2-4f64-9290-7651867428cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:22.718747 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600fd8c-13a2-4f64-9290-7651867428cc None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:22.719167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600fd8c-13a2-4f64-9290-7651867428cc None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:22.719570 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600fd8c-13a2-4f64-9290-7651867428cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:22.721279 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca5539-dc85-4561-aeef-dd8923e359e5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:22.734305 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4600fd8c-13a2-4f64-9290-7651867428cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:22.736189 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4600fd8c-13a2-4f64-9290-7651867428cc None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:22.748253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:22.759126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66dd565-6de5-45c0-a243-26b4aa5b77b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:22.759840 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dd565-6de5-45c0-a243-26b4aa5b77b4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:22.760126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dd565-6de5-45c0-a243-26b4aa5b77b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:22.761958 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dd565-6de5-45c0-a243-26b4aa5b77b4 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:22.761958 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66dd565-6de5-45c0-a243-26b4aa5b77b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:22.761958 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66dd565-6de5-45c0-a243-26b4aa5b77b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:22.762968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66dd565-6de5-45c0-a243-26b4aa5b77b4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:22.767347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-4600fd8c-13a2-4f64-9290-7651867428cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:22.767347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:22.767347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:22.767347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:22.767347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:22.767347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:22.767347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:22.767347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:22.767347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:22.767347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:22.767347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:22.767347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:22.767347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:22.767347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:22.767347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:22.768576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:22.768576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:22.768576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:22.768576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:22.768576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:22.768576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:22.768576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:22.768576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:22.768576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:22.768576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:22.768576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:22.768576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:22.768576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:22.768576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:22.768576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:22.768576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:22.768576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:22.769441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:22.769441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:22.769441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:22.769441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:22.770014 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 248/511] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:22.771282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600fd8c-13a2-4f64-9290-7651867428cc None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:22.785187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 264/512] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Jul 18 08:25:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:22.786051 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66dd565-6de5-45c0-a243-26b4aa5b77b4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:22.911407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66dd565-6de5-45c0-a243-26b4aa5b77b4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:22.914343 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600fd8c-13a2-4f64-9290-7651867428cc None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:22.937369 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:22.953034 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecc534-4ef4-415e-8fd8-33c01d9fbb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:22.956397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecc534-4ef4-415e-8fd8-33c01d9fbb35 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:22.956397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecc534-4ef4-415e-8fd8-33c01d9fbb35 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:22.956397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecc534-4ef4-415e-8fd8-33c01d9fbb35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:22.968709 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74ec06-49f6-4189-b018-213a16ec323e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:22.970616 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74ec06-49f6-4189-b018-213a16ec323e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:22.971085 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74ec06-49f6-4189-b018-213a16ec323e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:22.971542 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74ec06-49f6-4189-b018-213a16ec323e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:22.975063 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecc534-4ef4-415e-8fd8-33c01d9fbb35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:22.978145 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecc534-4ef4-415e-8fd8-33c01d9fbb35 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:22.993431 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74ec06-49f6-4189-b018-213a16ec323e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:22.995587 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74ec06-49f6-4189-b018-213a16ec323e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:23.010552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-30ecc534-4ef4-415e-8fd8-33c01d9fbb35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:23.010552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:23.010552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:23.010552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:23.010552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:23.010552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:23.010552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:23.010552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:23.010552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:23.010552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:23.010552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:23.010552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:23.010552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:23.010552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:23.010552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:23.012235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:23.012235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:23.012235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:23.012235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:23.012235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:23.012235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:23.012235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:23.012235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:23.012235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:23.012235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:23.012235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:23.012235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:23.012235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:23.012235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:23.012235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:23.012235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:23.012235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:23.014146 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:23.014146 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:23.014146 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:23.014146 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:23.014146 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 265/513] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:23.014146 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecc534-4ef4-415e-8fd8-33c01d9fbb35 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:23.027382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-7e74ec06-49f6-4189-b018-213a16ec323e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:23.027382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:23.027382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:23.027382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:23.027382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:23.027382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:23.027382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:23.027382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:23.027382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:23.027382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:23.027382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:23.027382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:23.027382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:23.027382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:23.027382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:23.029162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:23.029162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:23.029162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:23.029162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:23.029162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:23.029162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:23.029162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:23.029162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:23.029162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:23.029162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:23.029162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:23.029162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:23.029162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:23.029162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:23.029162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:23.029162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:23.029162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:23.031834 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:23.031834 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:23.031834 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:23.031834 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:23.031834 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 249/514] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:23.031834 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74ec06-49f6-4189-b018-213a16ec323e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:23.074314 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:23.161954 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecc534-4ef4-415e-8fd8-33c01d9fbb35 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:23.177312 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74ec06-49f6-4189-b018-213a16ec323e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:23.200428 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bccb631-cb93-4919-b1c5-8f65315aa01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:23.201097 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccb631-cb93-4919-b1c5-8f65315aa01d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:23.201379 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccb631-cb93-4919-b1c5-8f65315aa01d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:23.201656 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccb631-cb93-4919-b1c5-8f65315aa01d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:23.231577 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef17741-1a63-4083-9002-a54f093b860f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:23.232756 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef17741-1a63-4083-9002-a54f093b860f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:23.233242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef17741-1a63-4083-9002-a54f093b860f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:23.233742 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef17741-1a63-4083-9002-a54f093b860f None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:23.235137 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef17741-1a63-4083-9002-a54f093b860f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:23.235723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef17741-1a63-4083-9002-a54f093b860f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:23.238310 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef17741-1a63-4083-9002-a54f093b860f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:23.255111 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 250/515] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:23.256162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef17741-1a63-4083-9002-a54f093b860f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:23.287126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bccb631-cb93-4919-b1c5-8f65315aa01d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:23.289171 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:23.290435 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bccb631-cb93-4919-b1c5-8f65315aa01d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:23.331960 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:23.390569 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef17741-1a63-4083-9002-a54f093b860f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:23.448040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 266/516] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:25:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:25:23.448289 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693cd22-b8d1-4356-95c5-95fcec911ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:23.448433 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bccb631-cb93-4919-b1c5-8f65315aa01d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:23.448551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693cd22-b8d1-4356-95c5-95fcec911ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:23.448551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693cd22-b8d1-4356-95c5-95fcec911ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:23.448551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693cd22-b8d1-4356-95c5-95fcec911ea5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:23.554700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693cd22-b8d1-4356-95c5-95fcec911ea5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:23.556555 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693cd22-b8d1-4356-95c5-95fcec911ea5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:23.590902 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:23.594607 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bccb631-cb93-4919-b1c5-8f65315aa01d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:23.636302 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4abbc0-281f-4826-833c-3d562230a8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:23.637100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4abbc0-281f-4826-833c-3d562230a8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:23.637467 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4abbc0-281f-4826-833c-3d562230a8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:23.637830 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4abbc0-281f-4826-833c-3d562230a8f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:23.638686 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4abbc0-281f-4826-833c-3d562230a8f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:23.639087 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4abbc0-281f-4826-833c-3d562230a8f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:23.640672 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4abbc0-281f-4826-833c-3d562230a8f2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:23.642526 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:23.656791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 267/517] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:23.657930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4abbc0-281f-4826-833c-3d562230a8f2 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:23.687224 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 251/518] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:25:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:25:23.688289 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693cd22-b8d1-4356-95c5-95fcec911ea5 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:23.780486 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4abbc0-281f-4826-833c-3d562230a8f2 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:23.822383 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac667b-4834-473e-9e56-e4d72e4f16e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:23.823274 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac667b-4834-473e-9e56-e4d72e4f16e1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:23.823715 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac667b-4834-473e-9e56-e4d72e4f16e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:23.824159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac667b-4834-473e-9e56-e4d72e4f16e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:23.825028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac667b-4834-473e-9e56-e4d72e4f16e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:23.825472 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac667b-4834-473e-9e56-e4d72e4f16e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:23.827145 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac667b-4834-473e-9e56-e4d72e4f16e1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:23.838495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693cd22-b8d1-4356-95c5-95fcec911ea5 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:23.840544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 268/519] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:23.841963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac667b-4834-473e-9e56-e4d72e4f16e1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:23.876991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:23.887723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d4edf-a392-4db0-be22-d27cf6ba7a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:23.893449 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d4edf-a392-4db0-be22-d27cf6ba7a0a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:23.893449 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d4edf-a392-4db0-be22-d27cf6ba7a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:23.893449 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d4edf-a392-4db0-be22-d27cf6ba7a0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:23.900652 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d4edf-a392-4db0-be22-d27cf6ba7a0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:23.903204 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d4edf-a392-4db0-be22-d27cf6ba7a0a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:23.922870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-a77d4edf-a392-4db0-be22-d27cf6ba7a0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:23.922870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:23.922870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:23.922870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:23.922870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:23.922870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:23.922870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:23.922870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:23.922870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:23.922870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:23.922870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:23.922870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:23.922870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:25:23.922870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:25:23.922870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:25:23.924648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:25:23.924648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:23.924648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:23.924648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:23.924648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:23.924648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:23.924648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:23.924648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:23.924648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:23.924648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:23.924648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:23.924648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:23.924648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:23.924648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:23.924648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:23.924648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:23.924648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:23.926535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:25:23.926535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:25:23.926535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:25:23.926535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:25:23.926535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:25:23.926535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:25:23.926535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:23.926535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:23.926535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 252/520] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:23.926535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d4edf-a392-4db0-be22-d27cf6ba7a0a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:23.941864 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:24.005272 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac667b-4834-473e-9e56-e4d72e4f16e1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:24.049904 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bac5ff-a3cd-475e-9a1b-e584dd3be57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:24.050764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bac5ff-a3cd-475e-9a1b-e584dd3be57e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:24.051103 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bac5ff-a3cd-475e-9a1b-e584dd3be57e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:24.051443 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bac5ff-a3cd-475e-9a1b-e584dd3be57e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:24.058725 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d4edf-a392-4db0-be22-d27cf6ba7a0a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:24.065984 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bac5ff-a3cd-475e-9a1b-e584dd3be57e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:24.067867 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bac5ff-a3cd-475e-9a1b-e584dd3be57e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:24.090217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-29bac5ff-a3cd-475e-9a1b-e584dd3be57e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:24.090217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:24.090217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:24.090217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:24.090217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:24.090217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:24.090217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:24.090217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:24.090217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:24.090217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:24.090217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:24.090217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:24.090217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:24.090217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:24.090217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:24.091652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:24.091652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:24.091652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:24.091652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:24.091652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:24.091652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:24.091652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:24.091652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:24.091652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:24.091652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:24.091652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:24.091652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:24.091652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:24.091652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:24.091652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:24.091652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:24.091652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:24.094059 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:24.094059 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:24.094059 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:24.094059 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:24.094059 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 269/521] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:24.094059 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bac5ff-a3cd-475e-9a1b-e584dd3be57e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:24.108948 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e249e6fa-ed1a-4567-9679-73831b564de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:24.109813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249e6fa-ed1a-4567-9679-73831b564de4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:24.110532 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249e6fa-ed1a-4567-9679-73831b564de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:24.110532 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249e6fa-ed1a-4567-9679-73831b564de4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:24.119146 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e249e6fa-ed1a-4567-9679-73831b564de4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:24.120686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e249e6fa-ed1a-4567-9679-73831b564de4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:24.137168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-e249e6fa-ed1a-4567-9679-73831b564de4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 08:25:24.137168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:24.137168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:24.137168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:24.137168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:24.137168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:24.137168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:24.137168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:24.137168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:24.137168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:24.137168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:24.137168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:24.137168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:25:24.137168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:25:24.137168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:25:24.137168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:25:24.138492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:24.138492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:24.138492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:25:24.138492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:24.138492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:25:24.138492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:24.138492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:24.138492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:24.138492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:24.138492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:24.138492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:24.138492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:24.138492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:24.138492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:24.138492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:24.138492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:24.138492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:24.138492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:24.140366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:24.140366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:24.140366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:25:24.140366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:25:24.140366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:25:24.140366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:25:24.140366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:25:24.140366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:25:24.140366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 08:25:24.140366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:24.140366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 253/522] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:25:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:24.140366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e249e6fa-ed1a-4567-9679-73831b564de4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:24.198352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:24.229038 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bac5ff-a3cd-475e-9a1b-e584dd3be57e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:24.239435 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:24.258896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e249e6fa-ed1a-4567-9679-73831b564de4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:24.283451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a3b64-ed5a-46b7-82f4-091c056f50f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:24.284853 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a3b64-ed5a-46b7-82f4-091c056f50f1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:24.285338 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a3b64-ed5a-46b7-82f4-091c056f50f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:24.285829 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a3b64-ed5a-46b7-82f4-091c056f50f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:24.295112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5fbae-eea7-4654-b638-ac07fe7462a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:24.295814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5fbae-eea7-4654-b638-ac07fe7462a6 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:24.296180 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5fbae-eea7-4654-b638-ac07fe7462a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:24.296463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5fbae-eea7-4654-b638-ac07fe7462a6 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:24.297175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff5fbae-eea7-4654-b638-ac07fe7462a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:24.298631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff5fbae-eea7-4654-b638-ac07fe7462a6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:24.307871 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a3b64-ed5a-46b7-82f4-091c056f50f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:24.310471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a3b64-ed5a-46b7-82f4-091c056f50f1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:24.335441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-506a3b64-ed5a-46b7-82f4-091c056f50f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:24.335441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:24.335441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:24.335441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:24.335441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:24.335441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:24.335441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:24.335441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:24.335441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:24.335441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:24.335441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:24.335441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:24.335441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:24.335441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:24.335441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:24.336743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:24.336743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:24.336743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:24.336743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:24.336743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:24.336743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:24.336743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:24.336743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:24.336743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:24.336743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:24.336743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:24.336743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:24.336743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:24.336743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:24.336743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:24.336743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:24.336743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:24.337752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:24.337752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:24.337752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:24.337752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:24.337752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 270/523] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:24.338040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a3b64-ed5a-46b7-82f4-091c056f50f1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:24.377410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 254/524] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 18 08:25:24 2022] POST /identity/v3/users => generated 247 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:25:24.378263 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5fbae-eea7-4654-b638-ac07fe7462a6 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:24.470455 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a3b64-ed5a-46b7-82f4-091c056f50f1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:24.523212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d8f96-dee9-4a6b-85a8-3db1b1b44d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:24.523963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d8f96-dee9-4a6b-85a8-3db1b1b44d29 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:24.524253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d8f96-dee9-4a6b-85a8-3db1b1b44d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:24.524537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d8f96-dee9-4a6b-85a8-3db1b1b44d29 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:24.525501 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d8f96-dee9-4a6b-85a8-3db1b1b44d29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:24.525820 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d8f96-dee9-4a6b-85a8-3db1b1b44d29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:24.527387 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d8f96-dee9-4a6b-85a8-3db1b1b44d29 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:24.541568 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5fbae-eea7-4654-b638-ac07fe7462a6 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:24.545767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:24.559790 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 271/525] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:25:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:24.559790 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d8f96-dee9-4a6b-85a8-3db1b1b44d29 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:24.589362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:24.598171 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd567f15-cf30-4ac5-a3c3-5bbca7c88e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:24.598845 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd567f15-cf30-4ac5-a3c3-5bbca7c88e8d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:24.599180 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd567f15-cf30-4ac5-a3c3-5bbca7c88e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:24.599454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd567f15-cf30-4ac5-a3c3-5bbca7c88e8d None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:24.601468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd567f15-cf30-4ac5-a3c3-5bbca7c88e8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:24.601468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd567f15-cf30-4ac5-a3c3-5bbca7c88e8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:24.603218 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd567f15-cf30-4ac5-a3c3-5bbca7c88e8d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:24.623927 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 255/526] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:24.625126 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd567f15-cf30-4ac5-a3c3-5bbca7c88e8d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:24.703018 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d8f96-dee9-4a6b-85a8-3db1b1b44d29 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:24.748310 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ff631-8f80-4e76-8b9b-d34388599515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:24.749155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ff631-8f80-4e76-8b9b-d34388599515 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:24.749437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ff631-8f80-4e76-8b9b-d34388599515 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:24.749712 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ff631-8f80-4e76-8b9b-d34388599515 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:24.750573 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ff631-8f80-4e76-8b9b-d34388599515 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:24.750884 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ff631-8f80-4e76-8b9b-d34388599515 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:24.752413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ff631-8f80-4e76-8b9b-d34388599515 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:24.759559 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd567f15-cf30-4ac5-a3c3-5bbca7c88e8d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:24.782449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 272/527] 10.222.0.57 () {56 vars in 1139 bytes} [Mon Jul 18 08:25:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:24.783417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ff631-8f80-4e76-8b9b-d34388599515 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:24.807907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be588a-fef6-4c04-b086-8d1bdc0af100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:24.808760 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be588a-fef6-4c04-b086-8d1bdc0af100 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:24.809047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be588a-fef6-4c04-b086-8d1bdc0af100 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:24.809359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be588a-fef6-4c04-b086-8d1bdc0af100 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:24.810168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be588a-fef6-4c04-b086-8d1bdc0af100 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:24.810421 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be588a-fef6-4c04-b086-8d1bdc0af100 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:24.811991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be588a-fef6-4c04-b086-8d1bdc0af100 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:24.828987 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 256/528] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:24.828987 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be588a-fef6-4c04-b086-8d1bdc0af100 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:24.865228 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:24.901386 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:24.928815 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ff631-8f80-4e76-8b9b-d34388599515 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:24.966564 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be588a-fef6-4c04-b086-8d1bdc0af100 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:24.985593 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00bf89-65be-4c59-8c4d-6d40f912b85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:24.986272 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00bf89-65be-4c59-8c4d-6d40f912b85c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:24.986543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00bf89-65be-4c59-8c4d-6d40f912b85c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:24.986821 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00bf89-65be-4c59-8c4d-6d40f912b85c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:24.987694 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a00bf89-65be-4c59-8c4d-6d40f912b85c None admin] RBAC: Adding query filter params (role.id=9fc4c3dcb90945ebb4eb5566e62e7833, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=84cba474fc7c470ea119de697b9d7ef9) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:24.988018 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a00bf89-65be-4c59-8c4d-6d40f912b85c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:24.989471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a00bf89-65be-4c59-8c4d-6d40f912b85c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:24.997418 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 273/529] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:25:24 2022] GET /identity/v3/role_assignments?user.id=84cba474fc7c470ea119de697b9d7ef9&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=9fc4c3dcb90945ebb4eb5566e62e7833 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:24.997953 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00bf89-65be-4c59-8c4d-6d40f912b85c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:25.020934 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aee204-91e5-47b3-b3cb-5b748c1b7b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:25.022147 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aee204-91e5-47b3-b3cb-5b748c1b7b32 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:25.022759 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aee204-91e5-47b3-b3cb-5b748c1b7b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:25.023265 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aee204-91e5-47b3-b3cb-5b748c1b7b32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:25.032017 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aee204-91e5-47b3-b3cb-5b748c1b7b32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:25.033925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aee204-91e5-47b3-b3cb-5b748c1b7b32 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:25.057748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-f4aee204-91e5-47b3-b3cb-5b748c1b7b32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:25.057748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:25.057748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:25.057748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:25.057748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:25.057748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:25.057748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:25.057748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:25.057748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:25.057748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:25.057748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:25.057748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:25.057748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:25:25.057748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:25:25.057748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:25:25.058690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:25:25.058690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:25.058690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:25.058690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:25.058690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:25.058690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:25.058690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:25.058690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:25.058690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:25.058690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:25.058690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:25.058690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:25.058690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:25.058690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:25.058690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:25.058690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:25.058690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:25.059550 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:25:25.059550 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:25:25.059550 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:25:25.059550 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:25:25.059550 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:25:25.059550 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:25:25.059550 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:25.059550 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:25.062077 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 257/530] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:25.063201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aee204-91e5-47b3-b3cb-5b748c1b7b32 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:25.175082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:25.180001 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00bf89-65be-4c59-8c4d-6d40f912b85c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:25.199280 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aee204-91e5-47b3-b3cb-5b748c1b7b32 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:25.225354 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47194927-f807-4d9c-bbc3-e73d9f576b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:25.227038 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47194927-f807-4d9c-bbc3-e73d9f576b5d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:25.227544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47194927-f807-4d9c-bbc3-e73d9f576b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:25.228005 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47194927-f807-4d9c-bbc3-e73d9f576b5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:25.245476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c06c2-964f-449c-bbf9-843493b8d6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:25.246290 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c06c2-964f-449c-bbf9-843493b8d6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:25.246612 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c06c2-964f-449c-bbf9-843493b8d6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:25.246980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c06c2-964f-449c-bbf9-843493b8d6c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:25.251043 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:25.256032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c06c2-964f-449c-bbf9-843493b8d6c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:25.257630 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c06c2-964f-449c-bbf9-843493b8d6c8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:25.272573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-990c06c2-964f-449c-bbf9-843493b8d6c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:25.272573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:25.272573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:25.272573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:25.272573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:25.272573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:25.272573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:25.272573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:25.272573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:25.272573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:25.272573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:25.272573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:25.272573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:25:25.272573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:25:25.272573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:25:25.273578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:25:25.273578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:25.273578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:25.273578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:25.273578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:25.273578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:25.273578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:25.273578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:25.273578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:25.273578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:25.273578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:25.273578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:25.273578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:25.273578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:25.273578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:25.273578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:25.273578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:25.274589 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:25:25.274589 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:25:25.274589 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:25:25.274589 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:25:25.274589 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:25:25.274589 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:25:25.274589 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:25.274589 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:25.275469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 258/531] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:25.276087 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c06c2-964f-449c-bbf9-843493b8d6c8 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:25.342575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47194927-f807-4d9c-bbc3-e73d9f576b5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:25.344017 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47194927-f807-4d9c-bbc3-e73d9f576b5d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:25.406693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c06c2-964f-449c-bbf9-843493b8d6c8 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:25.462970 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 274/532] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:25:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:25:25.464061 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47194927-f807-4d9c-bbc3-e73d9f576b5d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:25.469613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0501e-9b57-466c-8e83-7d3cecfdca4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:25.470711 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0501e-9b57-466c-8e83-7d3cecfdca4b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:25.471367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0501e-9b57-466c-8e83-7d3cecfdca4b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:25.471832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0501e-9b57-466c-8e83-7d3cecfdca4b None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:25.473051 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0501e-9b57-466c-8e83-7d3cecfdca4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:25.473509 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0501e-9b57-466c-8e83-7d3cecfdca4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:25.475099 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0501e-9b57-466c-8e83-7d3cecfdca4b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:25.500198 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 259/533] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:25:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:25.501506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0501e-9b57-466c-8e83-7d3cecfdca4b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:25.506277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:25.515857 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:25.594230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47194927-f807-4d9c-bbc3-e73d9f576b5d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:25.659937 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60602c4c-331e-4841-8029-1d46edea6ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:25.661303 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60602c4c-331e-4841-8029-1d46edea6ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:25.662034 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60602c4c-331e-4841-8029-1d46edea6ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:25.662775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60602c4c-331e-4841-8029-1d46edea6ca5 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:25.664099 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0501e-9b57-466c-8e83-7d3cecfdca4b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:25.666062 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60602c4c-331e-4841-8029-1d46edea6ca5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:25.666859 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60602c4c-331e-4841-8029-1d46edea6ca5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:25.669941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60602c4c-331e-4841-8029-1d46edea6ca5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:25.687250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 275/534] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:25:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:25.688438 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60602c4c-331e-4841-8029-1d46edea6ca5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:25.713338 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0dfef6-b9d1-425e-9001-d546c5b63d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:25.714644 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dfef6-b9d1-425e-9001-d546c5b63d74 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:25.715412 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dfef6-b9d1-425e-9001-d546c5b63d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:25.715952 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dfef6-b9d1-425e-9001-d546c5b63d74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:25.730715 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0dfef6-b9d1-425e-9001-d546c5b63d74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:25.732677 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0dfef6-b9d1-425e-9001-d546c5b63d74 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:25.753993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-2f0dfef6-b9d1-425e-9001-d546c5b63d74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:25.753993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:25.753993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:25.753993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:25.753993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:25.753993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:25.753993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:25.753993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:25.753993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:25.753993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:25.753993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:25.753993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:25.753993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:25.753993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:25.753993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:25.754947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:25.754947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:25.754947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:25.754947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:25.754947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:25.754947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:25.754947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:25.754947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:25.754947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:25.754947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:25.754947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:25.754947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:25.754947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:25.754947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:25.754947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:25.754947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:25.754947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:25.755843 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:25.755843 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:25.755843 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:25.755843 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:25.759129 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 260/535] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:25.759514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0dfef6-b9d1-425e-9001-d546c5b63d74 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:25.822548 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:25.826634 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60602c4c-331e-4841-8029-1d46edea6ca5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:25.866930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cd515-a58b-44a9-909d-4d23ca4178f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:25.868000 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cd515-a58b-44a9-909d-4d23ca4178f6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:25.868516 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cd515-a58b-44a9-909d-4d23ca4178f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:25.868998 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cd515-a58b-44a9-909d-4d23ca4178f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:25.870082 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5cd515-a58b-44a9-909d-4d23ca4178f6 None admin] RBAC: Adding query filter params (role.id=9fc4c3dcb90945ebb4eb5566e62e7833, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=d5f9a129021445ae8154a5e6941291e0) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:25.870608 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5cd515-a58b-44a9-909d-4d23ca4178f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:25.872235 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:25.872565 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5cd515-a58b-44a9-909d-4d23ca4178f6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:25.879501 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 276/536] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:25:25 2022] GET /identity/v3/role_assignments?user.id=d5f9a129021445ae8154a5e6941291e0&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=9fc4c3dcb90945ebb4eb5566e62e7833 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:25.880780 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cd515-a58b-44a9-909d-4d23ca4178f6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:25.911879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0dfef6-b9d1-425e-9001-d546c5b63d74 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:25.974749 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdfaa6-2a0b-4930-8685-d57d6766a732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:25.975455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdfaa6-2a0b-4930-8685-d57d6766a732 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:25.975832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdfaa6-2a0b-4930-8685-d57d6766a732 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:25.976170 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdfaa6-2a0b-4930-8685-d57d6766a732 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:25.976940 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fdfaa6-2a0b-4930-8685-d57d6766a732 None admin] RBAC: Adding query filter params (role.id=9fc4c3dcb90945ebb4eb5566e62e7833, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=694088ed30b94bad9d3b16f3766252df) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:25.977253 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fdfaa6-2a0b-4930-8685-d57d6766a732 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:25.978828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fdfaa6-2a0b-4930-8685-d57d6766a732 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:25.992602 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 261/537] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:25:25 2022] GET /identity/v3/role_assignments?user.id=694088ed30b94bad9d3b16f3766252df&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=9fc4c3dcb90945ebb4eb5566e62e7833 => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:26.015448 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cd515-a58b-44a9-909d-4d23ca4178f6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:26.087034 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac64ab-3108-4b42-8475-ed69e654973c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:26.088739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac64ab-3108-4b42-8475-ed69e654973c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:26.089569 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac64ab-3108-4b42-8475-ed69e654973c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:26.090360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac64ab-3108-4b42-8475-ed69e654973c None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:26.092296 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac64ab-3108-4b42-8475-ed69e654973c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:26.092910 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac64ab-3108-4b42-8475-ed69e654973c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:26.096013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac64ab-3108-4b42-8475-ed69e654973c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:26.122741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 277/538] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:26.128024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdfaa6-2a0b-4930-8685-d57d6766a732 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:26.267329 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdfaa6-2a0b-4930-8685-d57d6766a732 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:26.291793 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:26.307559 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083b049-1757-477b-9156-b1e92537bcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:26.310060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083b049-1757-477b-9156-b1e92537bcf4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:26.310060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083b049-1757-477b-9156-b1e92537bcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:26.310060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083b049-1757-477b-9156-b1e92537bcf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:26.320303 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083b049-1757-477b-9156-b1e92537bcf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:26.323185 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083b049-1757-477b-9156-b1e92537bcf4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:26.348919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-5083b049-1757-477b-9156-b1e92537bcf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:26.348919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:26.348919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:26.348919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:26.348919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:26.348919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:26.348919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:26.348919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:26.348919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:26.348919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:26.348919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:26.348919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:26.348919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:25:26.348919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:25:26.348919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:25:26.349848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:25:26.349848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:26.349848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:26.349848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:26.349848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:26.349848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:26.349848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:26.349848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:26.349848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:26.349848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:26.349848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:26.349848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:26.349848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:26.349848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:26.349848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:26.349848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:26.349848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:26.350763 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:25:26.350763 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:25:26.350763 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:25:26.350763 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:25:26.350763 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:25:26.350763 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:25:26.350763 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:26.350763 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:26.356324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 262/539] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:26.365122 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac64ab-3108-4b42-8475-ed69e654973c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:26.453449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:26.622429 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac64ab-3108-4b42-8475-ed69e654973c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:26.699923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7675e-cf8c-4ef0-9c7c-d674c6229eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:26.699923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7675e-cf8c-4ef0-9c7c-d674c6229eed None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:26.699923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7675e-cf8c-4ef0-9c7c-d674c6229eed None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:26.699923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7675e-cf8c-4ef0-9c7c-d674c6229eed None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:26.700348 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7675e-cf8c-4ef0-9c7c-d674c6229eed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:26.700348 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7675e-cf8c-4ef0-9c7c-d674c6229eed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:26.700348 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7675e-cf8c-4ef0-9c7c-d674c6229eed None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:26.723650 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 278/540] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:25:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:26.735695 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083b049-1757-477b-9156-b1e92537bcf4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:26.753910 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9a082-b88d-4012-a04c-368e6a808121 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:26.754215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9a082-b88d-4012-a04c-368e6a808121 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:26.754504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9a082-b88d-4012-a04c-368e6a808121 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:26.760922 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 279/541] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:26.766297 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e699ec7-9965-40ee-beca-b0084380dd2c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:26.766595 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e699ec7-9965-40ee-beca-b0084380dd2c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:26.766879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e699ec7-9965-40ee-beca-b0084380dd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:26.819288 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-2e699ec7-9965-40ee-beca-b0084380dd2c None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:26.824486 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2e699ec7-9965-40ee-beca-b0084380dd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:27.076227 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083b049-1757-477b-9156-b1e92537bcf4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:27.114577 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 280/542] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:27.138051 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e699ec7-9965-40ee-beca-b0084380dd2c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:27.159193 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:27.220509 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db96c6e-665e-4fe5-910a-afe245fb884a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:27.221208 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db96c6e-665e-4fe5-910a-afe245fb884a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:27.221504 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db96c6e-665e-4fe5-910a-afe245fb884a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:27.221780 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db96c6e-665e-4fe5-910a-afe245fb884a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:27.222676 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db96c6e-665e-4fe5-910a-afe245fb884a None admin] RBAC: Adding query filter params (role.id=9fc4c3dcb90945ebb4eb5566e62e7833, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=24a6b794f5774e35b67e9ec4358ffc66) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:27.222991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db96c6e-665e-4fe5-910a-afe245fb884a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:27.224553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db96c6e-665e-4fe5-910a-afe245fb884a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:27.240101 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 263/543] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:25:26 2022] GET /identity/v3/role_assignments?user.id=24a6b794f5774e35b67e9ec4358ffc66&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=9fc4c3dcb90945ebb4eb5566e62e7833 => generated 259 bytes in 502 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:27.304995 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e699ec7-9965-40ee-beca-b0084380dd2c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:27.360319 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e94db-cd79-421a-a1b0-331980657696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:27.361056 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e94db-cd79-421a-a1b0-331980657696 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:27.361341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e94db-cd79-421a-a1b0-331980657696 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:27.361614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e94db-cd79-421a-a1b0-331980657696 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:27.380797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e94db-cd79-421a-a1b0-331980657696 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:27.389445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-077e94db-cd79-421a-a1b0-331980657696 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:27.389445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:27.389445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:27.389445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:27.389445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:27.389445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:27.389445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:27.389445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:27.389445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:27.389445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:27.389445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:27.389445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:27.389445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:27.389445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:27.389445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:27.389445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:27.390599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:27.390599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:27.390599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:27.390599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:27.390599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:27.390599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:27.390599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:27.390599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:27.390599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:27.390599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:27.390599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:27.390599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:27.390599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:27.390599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:27.390599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:27.390599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:27.391597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:27.391597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:27.391597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:27.391597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:27.391597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:27.391597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:27.391597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:27.391597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:27.391597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:27.391597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:27.391597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:27.391597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:27.395643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 281/544] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:25:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:25:27.397081 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db96c6e-665e-4fe5-910a-afe245fb884a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:27.486233 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:27.561229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db96c6e-665e-4fe5-910a-afe245fb884a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:27.615625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ddc6d-3938-402d-a98a-0ef25e300c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:27.616924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ddc6d-3938-402d-a98a-0ef25e300c15 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:27.617283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ddc6d-3938-402d-a98a-0ef25e300c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:27.617706 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ddc6d-3938-402d-a98a-0ef25e300c15 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:27.619095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ddc6d-3938-402d-a98a-0ef25e300c15 None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:27.619416 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ddc6d-3938-402d-a98a-0ef25e300c15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:27.620889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ddc6d-3938-402d-a98a-0ef25e300c15 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:27.635087 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 264/545] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:25:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:27.640775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e94db-cd79-421a-a1b0-331980657696 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:27.794558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:27.856792 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e94db-cd79-421a-a1b0-331980657696 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:27.934942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287915d-8035-4aea-9406-6d07206cf5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:27.936293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287915d-8035-4aea-9406-6d07206cf5fd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:27.936801 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287915d-8035-4aea-9406-6d07206cf5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:27.937105 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287915d-8035-4aea-9406-6d07206cf5fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:27.958325 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c287915d-8035-4aea-9406-6d07206cf5fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:27.961664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c287915d-8035-4aea-9406-6d07206cf5fd None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:27.994825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-c287915d-8035-4aea-9406-6d07206cf5fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:27.994825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:27.994825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:27.994825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:27.994825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:27.994825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:27.994825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:27.994825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:27.994825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:27.994825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:27.994825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:27.994825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:27.994825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:27.994825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:27.994825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:27.995778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:27.995778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:27.995778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:27.995778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:27.995778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:27.995778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:27.995778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:27.995778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:27.995778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:27.995778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:27.995778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:27.995778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:27.995778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:27.995778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:27.995778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:27.995778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:27.995778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:27.997163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:27.997163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:27.997163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:27.997163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:27.997932 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 282/546] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:25:28.004844 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ddc6d-3938-402d-a98a-0ef25e300c15 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:28.162312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a98e0-adf2-4944-9fda-78a323d16bf7 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:28.162617 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a98e0-adf2-4944-9fda-78a323d16bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:28.162875 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a98e0-adf2-4944-9fda-78a323d16bf7 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:28.164998 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 283/547] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:28.176494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948310e-1fe2-4a90-93bb-f69f48f61f49 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:28.176686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ddc6d-3938-402d-a98a-0ef25e300c15 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:28.176771 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948310e-1fe2-4a90-93bb-f69f48f61f49 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:28.177054 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948310e-1fe2-4a90-93bb-f69f48f61f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:28.185163 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:28.216487 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a37f1-670f-4bd9-9a9b-97a78dfa0c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:28.217172 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a37f1-670f-4bd9-9a9b-97a78dfa0c15 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:28.217453 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a37f1-670f-4bd9-9a9b-97a78dfa0c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:28.217724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a37f1-670f-4bd9-9a9b-97a78dfa0c15 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:28.218457 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a37f1-670f-4bd9-9a9b-97a78dfa0c15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:28.218762 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a37f1-670f-4bd9-9a9b-97a78dfa0c15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:28.220175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a37f1-670f-4bd9-9a9b-97a78dfa0c15 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:28.220826 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7948310e-1fe2-4a90-93bb-f69f48f61f49 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:28.222077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7948310e-1fe2-4a90-93bb-f69f48f61f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:28.235060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 265/548] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:28.241360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a37f1-670f-4bd9-9a9b-97a78dfa0c15 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:28.322565 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 284/549] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:25:28.347941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948310e-1fe2-4a90-93bb-f69f48f61f49 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:28.498623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:28.509631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a37f1-670f-4bd9-9a9b-97a78dfa0c15 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:28.595424 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948310e-1fe2-4a90-93bb-f69f48f61f49 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:28.598508 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e3b8b-380c-4d1f-862f-f0841ae92958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:28.599335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e3b8b-380c-4d1f-862f-f0841ae92958 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:28.599622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e3b8b-380c-4d1f-862f-f0841ae92958 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:28.599926 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e3b8b-380c-4d1f-862f-f0841ae92958 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:28.617214 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e3b8b-380c-4d1f-862f-f0841ae92958 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:28.618737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e3b8b-380c-4d1f-862f-f0841ae92958 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:28.652641 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd776210-a23c-4381-81a3-f0b2bdbc0104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:28.653459 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd776210-a23c-4381-81a3-f0b2bdbc0104 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:28.653738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-3c9e3b8b-380c-4d1f-862f-f0841ae92958 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 08:25:28.653738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:28.653738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:28.653738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:28.653738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:28.653738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:28.653738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:28.653738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:28.653738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:28.653738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:28.653738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:28.653738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:28.653738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:25:28.653738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:25:28.653738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:25:28.653738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:25:28.654746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:28.654746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:28.654746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:25:28.654746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:28.654746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:25:28.654746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:28.654746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:28.654746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:28.654746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:28.654746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:28.654746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:28.654746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:28.654746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:28.654746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:28.654746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:28.654746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:28.654746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:28.654746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:28.655807 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:28.655807 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:28.655807 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:25:28.655807 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:25:28.655807 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:25:28.655807 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:25:28.655807 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:25:28.655807 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:25:28.655807 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 08:25:28.655807 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:28.656344 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd776210-a23c-4381-81a3-f0b2bdbc0104 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:28.656344 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd776210-a23c-4381-81a3-f0b2bdbc0104 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:28.657060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 266/550] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:25:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:25:28.662348 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e3b8b-380c-4d1f-862f-f0841ae92958 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:28.689612 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd776210-a23c-4381-81a3-f0b2bdbc0104 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:28.697620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-fd776210-a23c-4381-81a3-f0b2bdbc0104 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:28.697620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:28.697620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:28.697620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:28.697620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:28.697620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:28.697620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:28.697620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:28.697620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:28.697620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:28.697620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:28.697620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:28.697620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:28.697620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:28.697620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:28.697620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:28.698596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:28.698596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:28.698596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:28.698596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:28.698596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:28.698596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:28.698596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:28.698596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:28.698596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:28.698596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:28.698596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:28.698596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:28.698596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:28.698596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:28.698596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:28.698596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:28.699449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:28.699449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:28.699449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:28.699449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:28.699449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:28.699449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:28.699449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:28.699449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:28.699449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:28.699449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:28.699449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:28.699449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:28.703874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 285/551] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:25:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:25:28.705126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd776210-a23c-4381-81a3-f0b2bdbc0104 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:28.754131 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:28.808111 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e3b8b-380c-4d1f-862f-f0841ae92958 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:28.856105 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0e9e1-a06b-47c4-8278-7200c2fc506f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:28.857318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0e9e1-a06b-47c4-8278-7200c2fc506f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:28.857607 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0e9e1-a06b-47c4-8278-7200c2fc506f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:28.857888 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0e9e1-a06b-47c4-8278-7200c2fc506f None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:28.859056 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0e9e1-a06b-47c4-8278-7200c2fc506f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:28.859373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0e9e1-a06b-47c4-8278-7200c2fc506f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:28.861677 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0e9e1-a06b-47c4-8278-7200c2fc506f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:28.914187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 267/552] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Jul 18 08:25:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:28.915497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd776210-a23c-4381-81a3-f0b2bdbc0104 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:28.916187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44345da9-62ec-4e84-b4db-4f2f930d31d0 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:28.916750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44345da9-62ec-4e84-b4db-4f2f930d31d0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:28.917233 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44345da9-62ec-4e84-b4db-4f2f930d31d0 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:28.918805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 268/553] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:28.921291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44345da9-62ec-4e84-b4db-4f2f930d31d0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:28.941987 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:28.967709 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea543ca-612b-4bdb-8146-2e6eaf921568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:28.967709 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea543ca-612b-4bdb-8146-2e6eaf921568 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:28.967709 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea543ca-612b-4bdb-8146-2e6eaf921568 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:28.967709 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea543ca-612b-4bdb-8146-2e6eaf921568 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:28.968152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea543ca-612b-4bdb-8146-2e6eaf921568 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:28.968152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea543ca-612b-4bdb-8146-2e6eaf921568 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:28.969442 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea543ca-612b-4bdb-8146-2e6eaf921568 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:28.980343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 286/554] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:25:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:28.981666 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082749a7-0cee-4af0-96bd-7502039d94bc None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:28.981957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082749a7-0cee-4af0-96bd-7502039d94bc None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:28.982244 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082749a7-0cee-4af0-96bd-7502039d94bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:29.005652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-082749a7-0cee-4af0-96bd-7502039d94bc None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:29.006898 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-082749a7-0cee-4af0-96bd-7502039d94bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:29.070012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:29.095701 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 287/555] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:29.095701 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac8395-e8cc-4688-9d86-ff30a27768b5 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:29.095701 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac8395-e8cc-4688-9d86-ff30a27768b5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:29.095701 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac8395-e8cc-4688-9d86-ff30a27768b5 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:29.095701 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 288/556] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:29.096517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac8395-e8cc-4688-9d86-ff30a27768b5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:29.106952 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44345da9-62ec-4e84-b4db-4f2f930d31d0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:29.172932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b472961-b313-4029-bb72-17c7f7841853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:29.174168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b472961-b313-4029-bb72-17c7f7841853 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:29.174585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b472961-b313-4029-bb72-17c7f7841853 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:29.175089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b472961-b313-4029-bb72-17c7f7841853 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:29.228083 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac8395-e8cc-4688-9d86-ff30a27768b5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:29.240695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:29.274305 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909613f-423d-49f4-acc9-1d92423e0cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:29.274305 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909613f-423d-49f4-acc9-1d92423e0cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:29.274636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909613f-423d-49f4-acc9-1d92423e0cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:29.274711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909613f-423d-49f4-acc9-1d92423e0cd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:29.284415 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b472961-b313-4029-bb72-17c7f7841853 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:29.286510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b472961-b313-4029-bb72-17c7f7841853 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:29.288500 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909613f-423d-49f4-acc9-1d92423e0cd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:29.289986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909613f-423d-49f4-acc9-1d92423e0cd1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:29.311378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-c909613f-423d-49f4-acc9-1d92423e0cd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:29.311378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:29.311378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:29.311378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:29.311378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:29.311378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:29.311378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:29.311378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:29.311378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:29.311378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:29.311378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:29.311378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:29.311378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:29.311378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:29.311378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:29.320381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:29.320381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:29.320381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:29.320381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:29.320381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:29.320381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:29.320381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:29.320381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:29.320381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:29.320381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:29.320381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:29.320381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:29.320381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:29.320381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:29.320381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:29.320381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:29.320381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:29.321863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:29.321863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:29.321863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:29.321863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:29.321863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 289/557] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:29.321863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40b24d-ee21-4431-be0d-1c3649152b87 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:29.321863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40b24d-ee21-4431-be0d-1c3649152b87 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:29.321863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40b24d-ee21-4431-be0d-1c3649152b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:29.338109 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0e40b24d-ee21-4431-be0d-1c3649152b87 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:29.339352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0e40b24d-ee21-4431-be0d-1c3649152b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:29.406207 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:29.426262 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 269/558] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:25:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 08:25:29.427253 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b472961-b313-4029-bb72-17c7f7841853 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:29.437245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 290/559] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:29.439111 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40b24d-ee21-4431-be0d-1c3649152b87 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:29.561262 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b472961-b313-4029-bb72-17c7f7841853 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:29.578003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:29.590361 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40b24d-ee21-4431-be0d-1c3649152b87 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:29.613970 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfa51b-f426-40a3-8c0c-50b36e37879c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:29.615203 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfa51b-f426-40a3-8c0c-50b36e37879c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:29.615851 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfa51b-f426-40a3-8c0c-50b36e37879c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:29.616802 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfa51b-f426-40a3-8c0c-50b36e37879c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:29.630034 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedfa51b-f426-40a3-8c0c-50b36e37879c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:29.633216 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4136fe6f-abf3-4a48-9ecc-d610e34d6296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:29.633996 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136fe6f-abf3-4a48-9ecc-d610e34d6296 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:29.634246 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136fe6f-abf3-4a48-9ecc-d610e34d6296 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:29.634549 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136fe6f-abf3-4a48-9ecc-d610e34d6296 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:29.635188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4136fe6f-abf3-4a48-9ecc-d610e34d6296 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:29.635496 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4136fe6f-abf3-4a48-9ecc-d610e34d6296 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:29.636935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4136fe6f-abf3-4a48-9ecc-d610e34d6296 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:29.639599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-fedfa51b-f426-40a3-8c0c-50b36e37879c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:29.639599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:29.639599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:29.639599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:29.639599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:29.639599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:29.639599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:29.639599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:29.639599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:29.639599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:29.639599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:29.639599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:29.639599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:29.639599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:29.639599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:29.639599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:29.640613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:29.640613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:29.640613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:29.640613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:29.640613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:29.640613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:29.640613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:29.640613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:29.640613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:29.640613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:29.640613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:29.640613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:29.640613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:29.640613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:29.640613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:29.640613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:29.641506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:29.641506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:29.641506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:29.641506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:29.641506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:29.641506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:29.641506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:29.641506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:29.641506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:29.641506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:29.641506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:29.641506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:29.641506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 270/560] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:25:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:29.642214 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfa51b-f426-40a3-8c0c-50b36e37879c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:29.655421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 291/561] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:29.656553 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4136fe6f-abf3-4a48-9ecc-d610e34d6296 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:29.722158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:29.787918 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4136fe6f-abf3-4a48-9ecc-d610e34d6296 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:29.824753 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfa51b-f426-40a3-8c0c-50b36e37879c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:29.838447 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a769d9-7b8a-4717-b1e1-26283c072b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:29.840070 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a769d9-7b8a-4717-b1e1-26283c072b33 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:29.840600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a769d9-7b8a-4717-b1e1-26283c072b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:29.841057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a769d9-7b8a-4717-b1e1-26283c072b33 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:29.862723 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a769d9-7b8a-4717-b1e1-26283c072b33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:29.877952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-c7a769d9-7b8a-4717-b1e1-26283c072b33 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:29.877952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:29.877952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:29.877952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:29.877952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:29.877952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:29.877952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:29.877952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:29.877952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:29.877952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:29.877952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:29.877952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:29.877952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:29.877952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:29.877952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:29.877952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:29.878990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:29.878990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:29.878990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:29.878990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:29.878990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:29.878990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:29.878990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:29.878990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:29.878990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:29.878990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:29.878990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:29.878990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:29.878990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:29.878990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:29.878990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:29.878990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:29.879925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:29.879925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:29.879925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:29.879925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:29.879925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:29.879925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:29.879925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:29.879925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:29.879925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:29.879925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:29.879925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:29.879925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:29.879925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 292/562] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:25:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:29.881330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a769d9-7b8a-4717-b1e1-26283c072b33 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:29.895943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa00e8-ff9d-4dd8-b51b-7a2f04463933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:29.897071 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa00e8-ff9d-4dd8-b51b-7a2f04463933 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:29.898125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa00e8-ff9d-4dd8-b51b-7a2f04463933 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:29.898292 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa00e8-ff9d-4dd8-b51b-7a2f04463933 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:29.901946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:29.913174 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa00e8-ff9d-4dd8-b51b-7a2f04463933 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:29.914678 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa00e8-ff9d-4dd8-b51b-7a2f04463933 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:29.934543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-27aa00e8-ff9d-4dd8-b51b-7a2f04463933 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:29.934543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:29.934543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:29.934543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:29.934543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:29.934543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:29.934543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:29.934543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:29.934543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:29.934543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:29.934543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:29.934543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:29.934543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:29.934543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:29.934543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:29.943757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:29.943757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:29.943757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:29.943757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:29.943757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:29.943757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:29.943757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:29.943757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:29.943757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:29.943757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:29.943757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:29.943757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:29.943757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:29.943757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:29.943757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:29.943757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:29.943757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:29.945920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:29.945920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:29.945920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:29.945920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:29.945920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 271/563] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:29.945920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa00e8-ff9d-4dd8-b51b-7a2f04463933 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:30.020332 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a769d9-7b8a-4717-b1e1-26283c072b33 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:30.059094 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3722c5e-3813-4aa2-b73c-7d4c4c9ae5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:30.059801 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3722c5e-3813-4aa2-b73c-7d4c4c9ae5cc None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:30.059970 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:30.060067 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3722c5e-3813-4aa2-b73c-7d4c4c9ae5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:30.060349 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3722c5e-3813-4aa2-b73c-7d4c4c9ae5cc None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:30.061111 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3722c5e-3813-4aa2-b73c-7d4c4c9ae5cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:30.061423 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3722c5e-3813-4aa2-b73c-7d4c4c9ae5cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:30.062820 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3722c5e-3813-4aa2-b73c-7d4c4c9ae5cc None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:30.066881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa00e8-ff9d-4dd8-b51b-7a2f04463933 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:30.072416 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 293/564] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:25:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:30.073279 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3722c5e-3813-4aa2-b73c-7d4c4c9ae5cc None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:30.108033 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e64ea-06b9-44c4-a2a7-f9e6dbf0b6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:30.108897 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e64ea-06b9-44c4-a2a7-f9e6dbf0b6cd None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:30.109277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e64ea-06b9-44c4-a2a7-f9e6dbf0b6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:30.109997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e64ea-06b9-44c4-a2a7-f9e6dbf0b6cd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:30.118731 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e64ea-06b9-44c4-a2a7-f9e6dbf0b6cd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:30.120335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e64ea-06b9-44c4-a2a7-f9e6dbf0b6cd None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:30.136279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-4f6e64ea-06b9-44c4-a2a7-f9e6dbf0b6cd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 08:25:30.136279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:30.136279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:30.136279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:30.136279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:30.136279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:30.136279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:30.136279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:30.136279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:30.136279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:30.136279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:30.136279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:30.136279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:25:30.136279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:25:30.136279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:25:30.136279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:25:30.138202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:30.138202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:30.138202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:25:30.138202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:30.138202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:25:30.138202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:30.138202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:30.138202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:30.138202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:30.138202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:30.138202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:30.138202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:30.138202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:30.138202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:30.138202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:30.138202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:30.138202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:30.138202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:30.140332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:30.140332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:30.140332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:25:30.140332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:25:30.140332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:25:30.140332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:25:30.140332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:25:30.140332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:25:30.140332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 08:25:30.140332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:30.140332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 272/565] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 18 08:25:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:30.140332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e64ea-06b9-44c4-a2a7-f9e6dbf0b6cd None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:30.166188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:30.213652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3722c5e-3813-4aa2-b73c-7d4c4c9ae5cc None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:30.254014 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f8fa3-f2ee-4d54-8667-f5db07c18822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:30.254708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f8fa3-f2ee-4d54-8667-f5db07c18822 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:30.254998 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f8fa3-f2ee-4d54-8667-f5db07c18822 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:30.255289 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f8fa3-f2ee-4d54-8667-f5db07c18822 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:30.256223 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f8fa3-f2ee-4d54-8667-f5db07c18822 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:30.256540 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f8fa3-f2ee-4d54-8667-f5db07c18822 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:30.258582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f8fa3-f2ee-4d54-8667-f5db07c18822 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:30.270231 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 294/566] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:25:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:30.270231 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171142c-1aa9-46ea-a74a-a8e026d59457 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:30.270558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171142c-1aa9-46ea-a74a-a8e026d59457 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:30.270802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171142c-1aa9-46ea-a74a-a8e026d59457 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:30.272648 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 295/567] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:30.273619 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171142c-1aa9-46ea-a74a-a8e026d59457 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:30.282920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e64ea-06b9-44c4-a2a7-f9e6dbf0b6cd None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:30.324310 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e6198-1fa7-46cd-82ab-c1a6704eab3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:30.324996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e6198-1fa7-46cd-82ab-c1a6704eab3b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:30.325279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e6198-1fa7-46cd-82ab-c1a6704eab3b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:30.325567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e6198-1fa7-46cd-82ab-c1a6704eab3b None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:30.326305 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e6198-1fa7-46cd-82ab-c1a6704eab3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:30.326613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e6198-1fa7-46cd-82ab-c1a6704eab3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:30.328056 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e6198-1fa7-46cd-82ab-c1a6704eab3b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:30.342054 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 273/568] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:30.342991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e6198-1fa7-46cd-82ab-c1a6704eab3b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:30.368226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:30.401550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171142c-1aa9-46ea-a74a-a8e026d59457 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:30.441175 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601bb1e2-a34b-49f7-9132-3668a2cbe8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:30.441980 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bb1e2-a34b-49f7-9132-3668a2cbe8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:30.442264 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bb1e2-a34b-49f7-9132-3668a2cbe8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:30.442542 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bb1e2-a34b-49f7-9132-3668a2cbe8d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:30.445889 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:30.457622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601bb1e2-a34b-49f7-9132-3668a2cbe8d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:30.459118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601bb1e2-a34b-49f7-9132-3668a2cbe8d7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:30.468154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e6198-1fa7-46cd-82ab-c1a6704eab3b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:30.480796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-601bb1e2-a34b-49f7-9132-3668a2cbe8d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:30.480796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:30.480796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:30.480796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:30.480796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:30.480796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:30.480796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:30.480796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:30.480796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:30.480796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:30.480796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:30.480796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:30.480796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:30.480796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:30.480796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:30.482460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:30.482460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:30.482460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:30.482460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:30.482460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:30.482460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:30.482460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:30.482460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:30.482460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:30.482460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:30.482460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:30.482460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:30.482460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:30.482460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:30.482460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:30.482460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:30.482460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:30.490151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:30.490151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:30.490151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:30.490151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:30.490151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 296/569] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:30.490151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601bb1e2-a34b-49f7-9132-3668a2cbe8d7 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:30.527308 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e28270-a895-4508-9673-0a6b064bb676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:30.528023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e28270-a895-4508-9673-0a6b064bb676 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:30.528305 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e28270-a895-4508-9673-0a6b064bb676 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:30.528577 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e28270-a895-4508-9673-0a6b064bb676 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:30.529437 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e28270-a895-4508-9673-0a6b064bb676 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:30.529744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e28270-a895-4508-9673-0a6b064bb676 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:30.531196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e28270-a895-4508-9673-0a6b064bb676 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:30.552123 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 274/570] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 18 08:25:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:30.553610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1f4a7-3152-40f5-96f4-cb1e729003ed None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:30.559889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1f4a7-3152-40f5-96f4-cb1e729003ed None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:30.559889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1f4a7-3152-40f5-96f4-cb1e729003ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:30.573874 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-1dc1f4a7-3152-40f5-96f4-cb1e729003ed None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:30.575106 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1dc1f4a7-3152-40f5-96f4-cb1e729003ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:30.617877 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601bb1e2-a34b-49f7-9132-3668a2cbe8d7 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:30.660291 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef09ad3-62b2-4403-8bf0-1f1c534e26c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:30.661010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef09ad3-62b2-4403-8bf0-1f1c534e26c5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:30.661293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef09ad3-62b2-4403-8bf0-1f1c534e26c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:30.661572 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef09ad3-62b2-4403-8bf0-1f1c534e26c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:30.668886 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 275/571] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:30.670431 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1f4a7-3152-40f5-96f4-cb1e729003ed None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:30.674215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef09ad3-62b2-4403-8bf0-1f1c534e26c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:30.674932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:30.675789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef09ad3-62b2-4403-8bf0-1f1c534e26c5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:30.697791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-0ef09ad3-62b2-4403-8bf0-1f1c534e26c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:30.697791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:30.697791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:30.697791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:30.697791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:30.697791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:30.697791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:30.697791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:30.697791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:30.697791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:30.697791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:30.697791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:30.697791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:30.697791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:30.697791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:30.699558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:30.699558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:30.699558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:30.699558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:30.699558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:30.699558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:30.699558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:30.699558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:30.699558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:30.699558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:30.699558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:30.699558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:30.699558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:30.699558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:30.699558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:30.699558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:30.699558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:30.701785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:30.701785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:30.701785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:30.701785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:30.703792 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 297/572] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:30.704859 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef09ad3-62b2-4403-8bf0-1f1c534e26c5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:30.764917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:30.818297 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1f4a7-3152-40f5-96f4-cb1e729003ed None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:30.866464 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a511a3-3988-4f89-9a29-b6bf21acdb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:30.867146 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a511a3-3988-4f89-9a29-b6bf21acdb47 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:30.867428 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a511a3-3988-4f89-9a29-b6bf21acdb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:30.867738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a511a3-3988-4f89-9a29-b6bf21acdb47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:30.872799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef09ad3-62b2-4403-8bf0-1f1c534e26c5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:30.875437 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a511a3-3988-4f89-9a29-b6bf21acdb47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:30.877191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a511a3-3988-4f89-9a29-b6bf21acdb47 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:30.893364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-96a511a3-3988-4f89-9a29-b6bf21acdb47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:30.893364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:30.893364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:30.893364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:30.893364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:30.893364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:30.893364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:30.893364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:30.893364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:30.893364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:30.893364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:30.893364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:30.893364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:25:30.893364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:25:30.893364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:25:30.894468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:25:30.894468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:30.894468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:30.894468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:30.894468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:30.894468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:30.894468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:30.894468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:30.894468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:30.894468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:30.894468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:30.894468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:30.894468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:30.894468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:30.894468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:30.894468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:30.894468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:30.895541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:25:30.895541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:25:30.895541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:25:30.895541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:25:30.895541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:25:30.895541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:25:30.895541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:30.895541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:30.896111 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 276/573] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:30.897065 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a511a3-3988-4f89-9a29-b6bf21acdb47 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:30.925412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76acb5aa-010e-494c-81fa-a3944d61ef34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:30.926442 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acb5aa-010e-494c-81fa-a3944d61ef34 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:30.926949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acb5aa-010e-494c-81fa-a3944d61ef34 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:30.927451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acb5aa-010e-494c-81fa-a3944d61ef34 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:30.929537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76acb5aa-010e-494c-81fa-a3944d61ef34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:30.930097 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76acb5aa-010e-494c-81fa-a3944d61ef34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:30.932414 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76acb5aa-010e-494c-81fa-a3944d61ef34 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:30.960661 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 298/574] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:30.960661 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76acb5aa-010e-494c-81fa-a3944d61ef34 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:30.973319 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:31.013665 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a511a3-3988-4f89-9a29-b6bf21acdb47 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:31.052862 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e266711-8bfa-4676-8fd7-1ccb84728ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:31.053872 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e266711-8bfa-4676-8fd7-1ccb84728ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:31.054863 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e266711-8bfa-4676-8fd7-1ccb84728ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:31.055145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e266711-8bfa-4676-8fd7-1ccb84728ff1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:31.066956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e266711-8bfa-4676-8fd7-1ccb84728ff1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:31.068638 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e266711-8bfa-4676-8fd7-1ccb84728ff1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:31.089027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-4e266711-8bfa-4676-8fd7-1ccb84728ff1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:31.089027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:31.089027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:31.089027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:31.089027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:31.089027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:31.089027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:31.089027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:31.089027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:31.089027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:31.089027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:31.089027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:31.089027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:31.089027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:31.089027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:31.090830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:31.090830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:31.090830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:31.090830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:31.090830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:31.090830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:31.090830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:31.090830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:31.090830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:31.090830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:31.090830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:31.090830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:31.090830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:31.090830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:31.090830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:31.090830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:31.090830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:31.092588 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:31.092588 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:31.092588 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:31.092588 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:31.094795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 277/575] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:31.095753 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e266711-8bfa-4676-8fd7-1ccb84728ff1 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:31.119557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:31.123605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76acb5aa-010e-494c-81fa-a3944d61ef34 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:31.162707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8eb1b-10c5-4eb0-9a2e-4ac26f7506ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:31.163465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8eb1b-10c5-4eb0-9a2e-4ac26f7506ac None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:31.163843 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8eb1b-10c5-4eb0-9a2e-4ac26f7506ac None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:31.164165 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df8eb1b-10c5-4eb0-9a2e-4ac26f7506ac None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:31.164951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8eb1b-10c5-4eb0-9a2e-4ac26f7506ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:31.165320 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8eb1b-10c5-4eb0-9a2e-4ac26f7506ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:31.166916 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df8eb1b-10c5-4eb0-9a2e-4ac26f7506ac None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:31.179407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 299/576] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:31.180294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8eb1b-10c5-4eb0-9a2e-4ac26f7506ac None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:31.244071 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e266711-8bfa-4676-8fd7-1ccb84728ff1 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:31.266092 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:31.290530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50ec5d-1eca-4429-b8d5-00b89980da39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:31.291429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50ec5d-1eca-4429-b8d5-00b89980da39 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:31.291847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50ec5d-1eca-4429-b8d5-00b89980da39 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:31.292189 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50ec5d-1eca-4429-b8d5-00b89980da39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:31.306254 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50ec5d-1eca-4429-b8d5-00b89980da39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:31.314852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-6b50ec5d-1eca-4429-b8d5-00b89980da39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:31.314852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:31.314852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:31.314852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:31.314852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:31.314852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:31.314852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:31.314852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:31.314852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:31.314852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:31.314852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:31.314852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:31.314852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:31.314852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:31.314852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:31.314852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:31.316765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:31.316765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:31.316765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:31.316765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:31.316765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:31.316765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:31.316765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:31.316765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:31.316765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:31.316765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:31.316765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:31.316765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:31.316765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:31.316765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:31.316765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:31.316765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:31.318707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:31.318707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:31.318707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:31.318707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:31.318707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:31.318707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:31.318707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:31.318707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:31.318707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:31.318707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:31.318707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:31.318707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:31.318707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 278/577] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:25:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:31.318707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50ec5d-1eca-4429-b8d5-00b89980da39 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:31.327765 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df8eb1b-10c5-4eb0-9a2e-4ac26f7506ac None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:31.375929 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42e544-1f9b-4501-a08f-b03c868b3bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:31.377068 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42e544-1f9b-4501-a08f-b03c868b3bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:31.377564 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42e544-1f9b-4501-a08f-b03c868b3bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:31.378068 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42e544-1f9b-4501-a08f-b03c868b3bf7 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:31.379574 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42e544-1f9b-4501-a08f-b03c868b3bf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:31.380165 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42e544-1f9b-4501-a08f-b03c868b3bf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:31.383451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42e544-1f9b-4501-a08f-b03c868b3bf7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:31.406465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 300/578] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:25:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:31.408090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42e544-1f9b-4501-a08f-b03c868b3bf7 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:31.433283 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:31.434324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50ec5d-1eca-4429-b8d5-00b89980da39 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:31.474469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34456e4b-e08f-4991-b786-b2029860eac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:31.476113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34456e4b-e08f-4991-b786-b2029860eac2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:31.476565 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34456e4b-e08f-4991-b786-b2029860eac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:31.476932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34456e4b-e08f-4991-b786-b2029860eac2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:31.486090 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34456e4b-e08f-4991-b786-b2029860eac2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:31.488009 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34456e4b-e08f-4991-b786-b2029860eac2 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:31.507221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-34456e4b-e08f-4991-b786-b2029860eac2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 08:25:31.507221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:31.507221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:31.507221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:31.507221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:31.507221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:31.507221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:31.507221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:31.507221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:31.507221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:31.507221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:31.507221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:31.507221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:25:31.507221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:25:31.507221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:25:31.507221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:25:31.509158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:31.509158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:31.509158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:25:31.509158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:31.509158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:25:31.509158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:31.509158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:31.509158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:31.509158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:31.509158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:31.509158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:31.509158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:31.509158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:31.509158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:31.509158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:31.509158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:31.509158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:31.509158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:31.510754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:31.510754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:31.510754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:25:31.510754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:25:31.510754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:25:31.510754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:25:31.510754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:25:31.510754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:25:31.510754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 08:25:31.510754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:31.510754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 279/579] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:25:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:31.510754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34456e4b-e08f-4991-b786-b2029860eac2 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:31.545983 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42e544-1f9b-4501-a08f-b03c868b3bf7 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:31.562982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:31.601927 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f7ff5-51ed-4fd4-857f-ea1e0556ed08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:31.602787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f7ff5-51ed-4fd4-857f-ea1e0556ed08 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:31.603178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f7ff5-51ed-4fd4-857f-ea1e0556ed08 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:31.603626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f7ff5-51ed-4fd4-857f-ea1e0556ed08 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:31.604738 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f7ff5-51ed-4fd4-857f-ea1e0556ed08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:31.605119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f7ff5-51ed-4fd4-857f-ea1e0556ed08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:31.606847 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f7ff5-51ed-4fd4-857f-ea1e0556ed08 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:31.623344 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 301/580] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:31.624795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f7ff5-51ed-4fd4-857f-ea1e0556ed08 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:31.662785 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34456e4b-e08f-4991-b786-b2029860eac2 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:31.716084 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422fd7d-b664-4c0b-8c4d-25764ab7b19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:31.717408 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422fd7d-b664-4c0b-8c4d-25764ab7b19c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:31.717878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422fd7d-b664-4c0b-8c4d-25764ab7b19c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:31.718251 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422fd7d-b664-4c0b-8c4d-25764ab7b19c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:31.728921 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422fd7d-b664-4c0b-8c4d-25764ab7b19c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:31.730486 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422fd7d-b664-4c0b-8c4d-25764ab7b19c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:31.739723 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:31.749414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-8422fd7d-b664-4c0b-8c4d-25764ab7b19c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 08:25:31.749414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:31.749414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:31.749414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:31.749414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:31.749414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:31.749414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:31.749414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:31.749414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:31.749414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:31.749414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:31.749414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:31.749414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:25:31.749414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:25:31.749414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:25:31.749414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:25:31.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:31.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:31.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:25:31.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:31.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:25:31.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:31.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:31.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:31.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:31.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:31.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:31.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:31.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:31.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:31.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:31.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:31.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:31.756058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:31.758458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:31.758458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:31.758458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:25:31.758458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:25:31.758458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:25:31.758458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:25:31.758458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:25:31.758458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:25:31.758458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 08:25:31.758458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:31.758458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 280/581] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:25:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:31.758458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422fd7d-b664-4c0b-8c4d-25764ab7b19c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:31.768983 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f7ff5-51ed-4fd4-857f-ea1e0556ed08 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:31.826557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae64b4-e8e1-4fac-89bf-a458ff585249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:31.827750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae64b4-e8e1-4fac-89bf-a458ff585249 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:31.828240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae64b4-e8e1-4fac-89bf-a458ff585249 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:31.828749 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae64b4-e8e1-4fac-89bf-a458ff585249 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:31.830208 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae64b4-e8e1-4fac-89bf-a458ff585249 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:31.830773 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae64b4-e8e1-4fac-89bf-a458ff585249 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:31.833405 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae64b4-e8e1-4fac-89bf-a458ff585249 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:31.851304 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 302/582] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:25:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:31.852627 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae64b4-e8e1-4fac-89bf-a458ff585249 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:31.871477 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:31.878439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422fd7d-b664-4c0b-8c4d-25764ab7b19c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:31.920126 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3c333-6664-43ca-9aa5-b4d062e20824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:31.921155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3c333-6664-43ca-9aa5-b4d062e20824 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:31.921634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3c333-6664-43ca-9aa5-b4d062e20824 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:31.922072 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3c333-6664-43ca-9aa5-b4d062e20824 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:31.923517 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3c333-6664-43ca-9aa5-b4d062e20824 None admin] RBAC: Adding query filter params (role.id=9fc4c3dcb90945ebb4eb5566e62e7833, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=09a5c2e804504038817cad76544dce5f) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:31.924052 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3c333-6664-43ca-9aa5-b4d062e20824 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:31.926580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3c333-6664-43ca-9aa5-b4d062e20824 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:31.947715 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 281/583] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:25:31 2022] GET /identity/v3/role_assignments?user.id=09a5c2e804504038817cad76544dce5f&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=9fc4c3dcb90945ebb4eb5566e62e7833 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:31.947715 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3c333-6664-43ca-9aa5-b4d062e20824 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:32.041595 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae64b4-e8e1-4fac-89bf-a458ff585249 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:32.089731 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694255d-53e0-4238-a5a4-56962a465131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:32.090623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694255d-53e0-4238-a5a4-56962a465131 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:32.091033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694255d-53e0-4238-a5a4-56962a465131 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:32.091432 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694255d-53e0-4238-a5a4-56962a465131 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:32.092901 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694255d-53e0-4238-a5a4-56962a465131 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:32.093287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694255d-53e0-4238-a5a4-56962a465131 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:32.094952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694255d-53e0-4238-a5a4-56962a465131 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:32.109264 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:32.122565 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 303/584] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Jul 18 08:25:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:32.123945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694255d-53e0-4238-a5a4-56962a465131 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:32.142425 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3c333-6664-43ca-9aa5-b4d062e20824 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:32.185783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc850b-6142-45e6-ac9f-ad0ffbaef01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:32.186683 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc850b-6142-45e6-ac9f-ad0ffbaef01c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:32.187114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc850b-6142-45e6-ac9f-ad0ffbaef01c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:32.187466 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc850b-6142-45e6-ac9f-ad0ffbaef01c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:32.196555 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc850b-6142-45e6-ac9f-ad0ffbaef01c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:32.198214 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc850b-6142-45e6-ac9f-ad0ffbaef01c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:32.214740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-04dc850b-6142-45e6-ac9f-ad0ffbaef01c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:32.214740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:32.214740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:32.214740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:32.214740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:32.214740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:32.214740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:32.214740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:32.214740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:32.214740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:32.214740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:32.214740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:32.214740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:25:32.214740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:25:32.214740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:25:32.216740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:25:32.216740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:32.216740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:32.216740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:32.216740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:32.216740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:32.216740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:32.216740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:32.216740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:32.216740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:32.216740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:32.216740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:32.216740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:32.216740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:32.216740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:32.216740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:32.216740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:32.227439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:25:32.227439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:25:32.227439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:25:32.227439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:25:32.227439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:25:32.227439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:25:32.227439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:32.227439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:32.227439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 282/585] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:32.227439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc850b-6142-45e6-ac9f-ad0ffbaef01c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:32.265397 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:32.317964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694255d-53e0-4238-a5a4-56962a465131 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:32.379007 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1162bc6-df24-444f-8f1e-e6adc18ebc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:32.380212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1162bc6-df24-444f-8f1e-e6adc18ebc94 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:32.380810 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1162bc6-df24-444f-8f1e-e6adc18ebc94 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:32.381209 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1162bc6-df24-444f-8f1e-e6adc18ebc94 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:32.383427 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1162bc6-df24-444f-8f1e-e6adc18ebc94 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:32.384129 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1162bc6-df24-444f-8f1e-e6adc18ebc94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:32.386900 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1162bc6-df24-444f-8f1e-e6adc18ebc94 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:32.417333 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc850b-6142-45e6-ac9f-ad0ffbaef01c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:32.421899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 304/586] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Jul 18 08:25:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:32.427705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1162bc6-df24-444f-8f1e-e6adc18ebc94 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:32.485975 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6f643-88b0-4cd0-953a-b0c136e7e989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:32.487315 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6f643-88b0-4cd0-953a-b0c136e7e989 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:32.487880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6f643-88b0-4cd0-953a-b0c136e7e989 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:32.488390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6f643-88b0-4cd0-953a-b0c136e7e989 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:32.508714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6f643-88b0-4cd0-953a-b0c136e7e989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:32.511719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6f643-88b0-4cd0-953a-b0c136e7e989 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:32.512696 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:32.548917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-0dd6f643-88b0-4cd0-953a-b0c136e7e989 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:32.548917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:32.548917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:32.548917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:32.548917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:32.548917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:32.548917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:32.548917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:32.548917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:32.548917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:32.548917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:32.548917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:32.548917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:32.548917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:32.548917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:32.549991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:32.549991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:32.549991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:32.549991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:32.549991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:32.549991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:32.549991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:32.549991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:32.549991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:32.549991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:32.549991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:32.549991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:32.549991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:32.549991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:32.549991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:32.549991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:32.549991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:32.550885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:32.550885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:32.550885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:32.550885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:32.551167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 283/587] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:32.552782 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6f643-88b0-4cd0-953a-b0c136e7e989 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:32.576714 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1162bc6-df24-444f-8f1e-e6adc18ebc94 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:32.626964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb6db8-5f24-4b64-92b0-32b38e09cd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:32.628245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb6db8-5f24-4b64-92b0-32b38e09cd18 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:32.628666 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb6db8-5f24-4b64-92b0-32b38e09cd18 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:32.629171 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb6db8-5f24-4b64-92b0-32b38e09cd18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:32.649793 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eb6db8-5f24-4b64-92b0-32b38e09cd18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:32.652536 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eb6db8-5f24-4b64-92b0-32b38e09cd18 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:32.659984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:32.678494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-17eb6db8-5f24-4b64-92b0-32b38e09cd18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:32.678494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:32.678494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:32.678494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:32.678494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:32.678494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:32.678494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:32.678494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:32.678494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:32.678494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:32.678494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:32.678494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:32.678494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:32.678494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:32.678494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:32.680614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:32.680614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:32.680614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:32.680614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:32.680614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:32.680614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:32.680614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:32.680614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:32.680614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:32.680614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:32.680614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:32.680614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:32.680614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:32.680614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:32.680614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:32.680614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:32.680614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:32.682202 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:32.682202 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:32.682202 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:32.682202 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:32.684524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 305/588] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:32.685489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb6db8-5f24-4b64-92b0-32b38e09cd18 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:32.701747 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6f643-88b0-4cd0-953a-b0c136e7e989 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:32.748689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0e4fe-3d63-4af4-aed3-1a772ace2af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:32.749354 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0e4fe-3d63-4af4-aed3-1a772ace2af9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:32.749634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0e4fe-3d63-4af4-aed3-1a772ace2af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:32.749907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0e4fe-3d63-4af4-aed3-1a772ace2af9 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:32.751210 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0e4fe-3d63-4af4-aed3-1a772ace2af9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:32.751210 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0e4fe-3d63-4af4-aed3-1a772ace2af9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:32.752499 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0e4fe-3d63-4af4-aed3-1a772ace2af9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:32.771710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 284/589] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:25:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:32.771710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0e4fe-3d63-4af4-aed3-1a772ace2af9 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:32.834537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb6db8-5f24-4b64-92b0-32b38e09cd18 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:32.850936 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:32.877449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25edc543-f81f-47b8-84f2-88c341280bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:32.878243 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25edc543-f81f-47b8-84f2-88c341280bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:32.878522 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25edc543-f81f-47b8-84f2-88c341280bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:32.878794 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25edc543-f81f-47b8-84f2-88c341280bb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:32.892637 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25edc543-f81f-47b8-84f2-88c341280bb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:32.894107 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25edc543-f81f-47b8-84f2-88c341280bb2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:32.915453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-25edc543-f81f-47b8-84f2-88c341280bb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:32.915453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:32.915453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:32.915453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:32.915453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:32.915453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:32.915453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:32.915453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:32.915453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:32.915453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:32.915453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:32.915453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:32.915453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:32.915453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:32.915453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:32.917531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:32.917531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:32.917531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:32.917531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:32.917531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:32.917531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:32.917531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:32.917531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:32.917531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:32.917531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:32.917531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:32.917531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:32.917531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:32.917531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:32.917531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:32.917531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:32.917531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:32.918759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:32.918759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:32.918759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:32.918759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:32.921344 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 306/590] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:32.922044 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25edc543-f81f-47b8-84f2-88c341280bb2 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:32.928547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0e4fe-3d63-4af4-aed3-1a772ace2af9 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:32.973481 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:32.980590 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a958940-8649-4218-a43c-9d35701c4cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:32.981262 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a958940-8649-4218-a43c-9d35701c4cdd None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:32.981571 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a958940-8649-4218-a43c-9d35701c4cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:32.986100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a958940-8649-4218-a43c-9d35701c4cdd None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:32.986100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a958940-8649-4218-a43c-9d35701c4cdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:32.986100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a958940-8649-4218-a43c-9d35701c4cdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:32.986100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a958940-8649-4218-a43c-9d35701c4cdd None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:32.999117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 285/591] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:32.999991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a958940-8649-4218-a43c-9d35701c4cdd None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:33.059491 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25edc543-f81f-47b8-84f2-88c341280bb2 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:33.099980 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3559df-b356-4286-911d-bc7cd8b00a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:33.101317 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3559df-b356-4286-911d-bc7cd8b00a43 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:33.101726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3559df-b356-4286-911d-bc7cd8b00a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:33.102116 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3559df-b356-4286-911d-bc7cd8b00a43 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:33.102638 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3559df-b356-4286-911d-bc7cd8b00a43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:33.102959 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3559df-b356-4286-911d-bc7cd8b00a43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:33.104402 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3559df-b356-4286-911d-bc7cd8b00a43 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:33.119940 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 307/592] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:33.121311 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3559df-b356-4286-911d-bc7cd8b00a43 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:33.126170 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a958940-8649-4218-a43c-9d35701c4cdd None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:33.150767 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:33.174913 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83090c8c-f744-4d62-b8bb-05359dcc2de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:33.175916 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83090c8c-f744-4d62-b8bb-05359dcc2de7 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:33.176155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83090c8c-f744-4d62-b8bb-05359dcc2de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:33.176406 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83090c8c-f744-4d62-b8bb-05359dcc2de7 None admin] PATH_INFO: `/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/84cba474fc7c470ea119de697b9d7ef9/roles/9fc4c3dcb90945ebb4eb5566e62e7833` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:33.196173 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83090c8c-f744-4d62-b8bb-05359dcc2de7 None admin] RBAC: Authorizing `identity:create_grant(project_id=35c846931fd443fbb83a6fbe0aeaa866, user_id=84cba474fc7c470ea119de697b9d7ef9, role_id=9fc4c3dcb90945ebb4eb5566e62e7833)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:33.198141 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83090c8c-f744-4d62-b8bb-05359dcc2de7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:33.248129 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3559df-b356-4286-911d-bc7cd8b00a43 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:33.285044 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd5556-50cd-4072-a889-976ed602cf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:33.285717 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd5556-50cd-4072-a889-976ed602cf83 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:33.286000 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd5556-50cd-4072-a889-976ed602cf83 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:33.286301 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd5556-50cd-4072-a889-976ed602cf83 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:33.287003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd5556-50cd-4072-a889-976ed602cf83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:33.287307 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd5556-50cd-4072-a889-976ed602cf83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:33.289360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd5556-50cd-4072-a889-976ed602cf83 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:33.311136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 308/593] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:33.311136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd5556-50cd-4072-a889-976ed602cf83 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:33.318941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 286/594] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:25:32 2022] PUT /identity/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/84cba474fc7c470ea119de697b9d7ef9/roles/9fc4c3dcb90945ebb4eb5566e62e7833 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:25:33.323586 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83090c8c-f744-4d62-b8bb-05359dcc2de7 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:33.399659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:33.443449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:33.469945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd5556-50cd-4072-a889-976ed602cf83 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:33.513752 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83090c8c-f744-4d62-b8bb-05359dcc2de7 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:33.545218 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dec718-681d-44b3-bcc1-b76f11390322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:33.546815 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dec718-681d-44b3-bcc1-b76f11390322 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:33.547397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dec718-681d-44b3-bcc1-b76f11390322 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:33.548085 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dec718-681d-44b3-bcc1-b76f11390322 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:33.559649 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dec718-681d-44b3-bcc1-b76f11390322 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:33.562358 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dec718-681d-44b3-bcc1-b76f11390322 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:33.580166 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0a86c-2b62-4aeb-88e3-a9b89bd4590b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:33.581061 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0a86c-2b62-4aeb-88e3-a9b89bd4590b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:33.581393 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0a86c-2b62-4aeb-88e3-a9b89bd4590b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:33.581872 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0a86c-2b62-4aeb-88e3-a9b89bd4590b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:33.584247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-43dec718-681d-44b3-bcc1-b76f11390322 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 08:25:33.584247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:33.584247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:33.584247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:33.584247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:33.584247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:33.584247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:33.584247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:33.584247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:33.584247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:33.584247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:33.584247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:33.584247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:25:33.584247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:25:33.584247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:25:33.584247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:25:33.585804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:33.585804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:33.585804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:25:33.585804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:33.585804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:25:33.585804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:33.585804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:33.585804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:33.585804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:33.585804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:33.585804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:33.585804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:33.585804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:33.585804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:33.585804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:33.585804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:33.585804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:33.585804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:33.586750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:33.586750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:33.586750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:25:33.586750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:25:33.586750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:25:33.586750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:25:33.586750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:25:33.586750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:25:33.586750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 08:25:33.586750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:33.586750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 309/595] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:25:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:33.587333 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dec718-681d-44b3-bcc1-b76f11390322 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:33.595932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0a86c-2b62-4aeb-88e3-a9b89bd4590b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:33.597586 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0a86c-2b62-4aeb-88e3-a9b89bd4590b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:33.617604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-c0c0a86c-2b62-4aeb-88e3-a9b89bd4590b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:33.617604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:33.617604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:33.617604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:33.617604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:33.617604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:33.617604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:33.617604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:33.617604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:33.617604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:33.617604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:33.617604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:33.617604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:25:33.617604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:25:33.617604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:25:33.618732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:25:33.618732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:33.618732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:33.618732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:33.618732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:33.618732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:33.618732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:33.618732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:33.618732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:33.618732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:33.618732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:33.618732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:33.618732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:33.618732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:33.618732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:33.618732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:33.618732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:33.620622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:25:33.620622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:25:33.620622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:25:33.620622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:25:33.620622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:25:33.620622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:25:33.620622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:33.620622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:33.620622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 287/596] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:33.620622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0a86c-2b62-4aeb-88e3-a9b89bd4590b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:33.714089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dec718-681d-44b3-bcc1-b76f11390322 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:33.754141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e41040-5dcb-4b4d-a3dd-ecd8f7cd0877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:33.755060 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e41040-5dcb-4b4d-a3dd-ecd8f7cd0877 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:33.755437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e41040-5dcb-4b4d-a3dd-ecd8f7cd0877 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:33.755776 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e41040-5dcb-4b4d-a3dd-ecd8f7cd0877 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:33.764540 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e41040-5dcb-4b4d-a3dd-ecd8f7cd0877 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:33.766025 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e41040-5dcb-4b4d-a3dd-ecd8f7cd0877 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:33.769099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:33.770614 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0a86c-2b62-4aeb-88e3-a9b89bd4590b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:33.775105 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:33.783298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-a1e41040-5dcb-4b4d-a3dd-ecd8f7cd0877 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:33.783298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:33.783298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:33.783298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:33.783298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:33.783298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:33.783298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:33.783298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:33.783298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:33.783298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:33.783298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:33.783298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:33.783298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:25:33.783298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:25:33.783298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:25:33.785930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:25:33.785930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:33.785930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:33.785930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:33.785930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:33.785930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:33.785930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:33.785930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:33.785930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:33.785930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:33.785930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:33.785930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:33.785930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:33.785930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:33.785930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:33.785930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:33.785930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:33.788450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:25:33.788450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:25:33.788450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:25:33.788450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:25:33.788450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:25:33.788450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:25:33.788450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:33.788450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:33.788450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 310/597] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:33.788450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e41040-5dcb-4b4d-a3dd-ecd8f7cd0877 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:33.847855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47bc52-e453-4984-9f18-20e5762c2e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:33.849017 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47bc52-e453-4984-9f18-20e5762c2e36 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:33.849509 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47bc52-e453-4984-9f18-20e5762c2e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:33.849979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47bc52-e453-4984-9f18-20e5762c2e36 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:33.855101 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47bc52-e453-4984-9f18-20e5762c2e36 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:33.855963 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47bc52-e453-4984-9f18-20e5762c2e36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:33.858052 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47bc52-e453-4984-9f18-20e5762c2e36 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:33.889773 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 288/598] 10.222.0.57 () {56 vars in 1139 bytes} [Mon Jul 18 08:25:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:33.891310 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47bc52-e453-4984-9f18-20e5762c2e36 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:33.902986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e41040-5dcb-4b4d-a3dd-ecd8f7cd0877 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:33.942322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847710f-031e-4ff9-9834-c430bb121bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:33.943061 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847710f-031e-4ff9-9834-c430bb121bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:33.943355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847710f-031e-4ff9-9834-c430bb121bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:33.943637 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847710f-031e-4ff9-9834-c430bb121bd1 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:33.944574 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847710f-031e-4ff9-9834-c430bb121bd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:33.945043 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847710f-031e-4ff9-9834-c430bb121bd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:33.946595 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847710f-031e-4ff9-9834-c430bb121bd1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:33.965970 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 311/599] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:25:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:33.965970 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847710f-031e-4ff9-9834-c430bb121bd1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:34.029925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47bc52-e453-4984-9f18-20e5762c2e36 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:34.042506 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:34.088225 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcd3f4-15d9-4bc0-bec6-4ad6bdf5697c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:34.089060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcd3f4-15d9-4bc0-bec6-4ad6bdf5697c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:34.089318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcd3f4-15d9-4bc0-bec6-4ad6bdf5697c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:34.089592 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcd3f4-15d9-4bc0-bec6-4ad6bdf5697c None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:34.090554 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcd3f4-15d9-4bc0-bec6-4ad6bdf5697c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:34.090863 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcd3f4-15d9-4bc0-bec6-4ad6bdf5697c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:34.092470 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcd3f4-15d9-4bc0-bec6-4ad6bdf5697c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:34.098877 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847710f-031e-4ff9-9834-c430bb121bd1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:34.106997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 289/600] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:25:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:34.107878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcd3f4-15d9-4bc0-bec6-4ad6bdf5697c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:34.149543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:34.168936 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647012bd-3995-4749-8618-a6ec6c414188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:34.169783 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647012bd-3995-4749-8618-a6ec6c414188 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:34.170043 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647012bd-3995-4749-8618-a6ec6c414188 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:34.170387 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647012bd-3995-4749-8618-a6ec6c414188 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:34.183330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647012bd-3995-4749-8618-a6ec6c414188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:34.184845 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647012bd-3995-4749-8618-a6ec6c414188 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:34.207789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-647012bd-3995-4749-8618-a6ec6c414188 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:34.207789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:34.207789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:34.207789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:34.207789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:34.207789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:34.207789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:34.207789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:34.207789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:34.207789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:34.207789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:34.207789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:34.207789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:34.207789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:34.207789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:34.208759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:34.208759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:34.208759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:34.208759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:34.208759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:34.208759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:34.208759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:34.208759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:34.208759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:34.208759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:34.208759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:34.208759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:34.208759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:34.208759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:34.208759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:34.208759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:34.208759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:34.210305 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:34.210305 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:34.210305 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:34.210305 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:34.210305 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 312/601] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:34.215572 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbeab5-7def-4dcd-8b75-7229a94d3d93 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:34.215879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbeab5-7def-4dcd-8b75-7229a94d3d93 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:34.216218 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbeab5-7def-4dcd-8b75-7229a94d3d93 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:34.218171 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 313/602] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:34.218300 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbeab5-7def-4dcd-8b75-7229a94d3d93 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:34.256336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcd3f4-15d9-4bc0-bec6-4ad6bdf5697c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:34.308643 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e627191-bd3b-4f52-bac3-ec67498b029d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:34.309575 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e627191-bd3b-4f52-bac3-ec67498b029d None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:34.309864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e627191-bd3b-4f52-bac3-ec67498b029d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:34.310169 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e627191-bd3b-4f52-bac3-ec67498b029d None admin] PATH_INFO: `/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/694088ed30b94bad9d3b16f3766252df/roles/9fc4c3dcb90945ebb4eb5566e62e7833` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:34.328916 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e627191-bd3b-4f52-bac3-ec67498b029d None admin] RBAC: Authorizing `identity:create_grant(project_id=35c846931fd443fbb83a6fbe0aeaa866, user_id=694088ed30b94bad9d3b16f3766252df, role_id=9fc4c3dcb90945ebb4eb5566e62e7833)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:34.330377 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e627191-bd3b-4f52-bac3-ec67498b029d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:34.358626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbeab5-7def-4dcd-8b75-7229a94d3d93 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:34.373765 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:34.399678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96414c-cec4-48db-9f33-162b51d9b9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:34.400605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96414c-cec4-48db-9f33-162b51d9b9e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:34.400888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96414c-cec4-48db-9f33-162b51d9b9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:34.401167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96414c-cec4-48db-9f33-162b51d9b9e7 None admin] PATH_INFO: `/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/d5f9a129021445ae8154a5e6941291e0/roles/9fc4c3dcb90945ebb4eb5566e62e7833` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:34.417899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96414c-cec4-48db-9f33-162b51d9b9e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=35c846931fd443fbb83a6fbe0aeaa866, user_id=d5f9a129021445ae8154a5e6941291e0, role_id=9fc4c3dcb90945ebb4eb5566e62e7833)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:34.419356 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96414c-cec4-48db-9f33-162b51d9b9e7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:34.433205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 290/603] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:25:34 2022] PUT /identity/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/694088ed30b94bad9d3b16f3766252df/roles/9fc4c3dcb90945ebb4eb5566e62e7833 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:25:34.434359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e627191-bd3b-4f52-bac3-ec67498b029d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:34.526286 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 314/604] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:25:34 2022] PUT /identity/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/d5f9a129021445ae8154a5e6941291e0/roles/9fc4c3dcb90945ebb4eb5566e62e7833 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:25:34.529658 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c563d5d-7c13-4e23-afc4-6049f9b2c487 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:34.530529 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c563d5d-7c13-4e23-afc4-6049f9b2c487 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:34.531031 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c563d5d-7c13-4e23-afc4-6049f9b2c487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:34.558470 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:34.565646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-6c563d5d-7c13-4e23-afc4-6049f9b2c487 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:34.568328 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6c563d5d-7c13-4e23-afc4-6049f9b2c487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:34.643710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e627191-bd3b-4f52-bac3-ec67498b029d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:34.692810 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4d77f-3aa5-476a-a00b-87e1eca7cabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:34.693654 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4d77f-3aa5-476a-a00b-87e1eca7cabc None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:34.693936 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4d77f-3aa5-476a-a00b-87e1eca7cabc None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:34.694449 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4d77f-3aa5-476a-a00b-87e1eca7cabc None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:34.695440 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4d77f-3aa5-476a-a00b-87e1eca7cabc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:34.695793 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4d77f-3aa5-476a-a00b-87e1eca7cabc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:34.697256 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4d77f-3aa5-476a-a00b-87e1eca7cabc None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:34.713457 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 291/605] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:34.718393 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4d77f-3aa5-476a-a00b-87e1eca7cabc None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:34.736741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 315/606] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:34.761751 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c563d5d-7c13-4e23-afc4-6049f9b2c487 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:34.870871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4d77f-3aa5-476a-a00b-87e1eca7cabc None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:34.906892 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:34.909330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:34.918846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53dfdc1-5eb6-4a22-99e8-43ce84d29bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:34.920092 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53dfdc1-5eb6-4a22-99e8-43ce84d29bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:34.921617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53dfdc1-5eb6-4a22-99e8-43ce84d29bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:34.922099 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53dfdc1-5eb6-4a22-99e8-43ce84d29bb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:34.922447 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c563d5d-7c13-4e23-afc4-6049f9b2c487 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:34.932224 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53dfdc1-5eb6-4a22-99e8-43ce84d29bb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:34.933923 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53dfdc1-5eb6-4a22-99e8-43ce84d29bb9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:34.953190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-b53dfdc1-5eb6-4a22-99e8-43ce84d29bb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:34.953190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:34.953190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:34.953190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:34.953190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:34.953190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:34.953190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:34.953190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:34.953190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:34.953190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:34.953190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:34.953190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:34.953190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:25:34.953190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:25:34.953190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:25:34.961622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:25:34.961622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:34.961622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:34.961622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:34.961622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:34.961622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:34.961622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:34.961622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:34.961622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:34.961622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:34.961622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:34.961622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:34.961622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:34.961622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:34.961622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:34.961622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:34.961622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:34.962705 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:25:34.962705 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:25:34.962705 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:25:34.962705 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:25:34.962705 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:25:34.962705 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:25:34.962705 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:34.962705 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:34.962705 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 292/607] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:34.964357 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53dfdc1-5eb6-4a22-99e8-43ce84d29bb9 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:34.990648 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9e97e-ac7f-4d62-aefc-b4d132eaf6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:34.991776 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9e97e-ac7f-4d62-aefc-b4d132eaf6df None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:34.992212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9e97e-ac7f-4d62-aefc-b4d132eaf6df None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:34.992634 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9e97e-ac7f-4d62-aefc-b4d132eaf6df None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:35.019020 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9e97e-ac7f-4d62-aefc-b4d132eaf6df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:35.030683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-85f9e97e-ac7f-4d62-aefc-b4d132eaf6df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:35.030683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:35.030683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:35.030683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:35.030683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:35.030683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:35.030683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:35.030683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:35.030683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:35.030683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:35.030683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:35.030683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:35.030683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:35.030683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:35.030683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:35.030683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:35.031935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:35.031935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:35.031935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:35.031935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:35.031935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:35.031935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:35.031935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:35.031935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:35.031935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:35.031935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:35.031935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:35.031935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:35.031935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:35.031935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:35.031935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:35.031935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:35.032830 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:35.032830 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:35.032830 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:35.032830 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:35.032830 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:35.032830 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:35.032830 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:35.032830 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:35.032830 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:35.032830 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:35.032830 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:35.032830 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:35.052188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 316/608] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:25:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:35.058918 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9e97e-ac7f-4d62-aefc-b4d132eaf6df None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:35.128625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53dfdc1-5eb6-4a22-99e8-43ce84d29bb9 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:35.180310 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713418bb-c7ed-4260-9ae4-75dcc92b8ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:35.181146 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713418bb-c7ed-4260-9ae4-75dcc92b8ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:35.181532 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713418bb-c7ed-4260-9ae4-75dcc92b8ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:35.181908 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713418bb-c7ed-4260-9ae4-75dcc92b8ec0 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:35.182823 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713418bb-c7ed-4260-9ae4-75dcc92b8ec0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:35.183238 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713418bb-c7ed-4260-9ae4-75dcc92b8ec0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:35.184880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713418bb-c7ed-4260-9ae4-75dcc92b8ec0 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:35.206358 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 293/609] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:25:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:35.219125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713418bb-c7ed-4260-9ae4-75dcc92b8ec0 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:35.308209 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9e97e-ac7f-4d62-aefc-b4d132eaf6df None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:35.369738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:35.418412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ffcfb-dc8f-4d76-953d-1e76dccc8385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:35.419435 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ffcfb-dc8f-4d76-953d-1e76dccc8385 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:35.419962 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ffcfb-dc8f-4d76-953d-1e76dccc8385 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:35.420460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ffcfb-dc8f-4d76-953d-1e76dccc8385 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:35.421993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ffcfb-dc8f-4d76-953d-1e76dccc8385 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:35.422468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ffcfb-dc8f-4d76-953d-1e76dccc8385 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:35.425609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ffcfb-dc8f-4d76-953d-1e76dccc8385 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:35.447131 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 317/610] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:25:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:35.453151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ffcfb-dc8f-4d76-953d-1e76dccc8385 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:35.503079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713418bb-c7ed-4260-9ae4-75dcc92b8ec0 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:35.511016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:35.578802 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea25cc3-940f-4187-8844-a60bf6b90a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:35.580940 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea25cc3-940f-4187-8844-a60bf6b90a70 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:35.581455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea25cc3-940f-4187-8844-a60bf6b90a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:35.581866 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea25cc3-940f-4187-8844-a60bf6b90a70 None admin] PATH_INFO: `/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/24a6b794f5774e35b67e9ec4358ffc66/roles/9fc4c3dcb90945ebb4eb5566e62e7833` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:35.614998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea25cc3-940f-4187-8844-a60bf6b90a70 None admin] RBAC: Authorizing `identity:create_grant(project_id=35c846931fd443fbb83a6fbe0aeaa866, user_id=24a6b794f5774e35b67e9ec4358ffc66, role_id=9fc4c3dcb90945ebb4eb5566e62e7833)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:35.618453 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea25cc3-940f-4187-8844-a60bf6b90a70 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:35.698665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ffcfb-dc8f-4d76-953d-1e76dccc8385 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:35.720954 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 294/611] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:25:35 2022] PUT /identity/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/24a6b794f5774e35b67e9ec4358ffc66/roles/9fc4c3dcb90945ebb4eb5566e62e7833 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:25:35.747887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c50c36-87f1-4375-b175-d7385cf170ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:35.748704 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c50c36-87f1-4375-b175-d7385cf170ac None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:35.748991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c50c36-87f1-4375-b175-d7385cf170ac None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:35.749272 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c50c36-87f1-4375-b175-d7385cf170ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:35.763424 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c50c36-87f1-4375-b175-d7385cf170ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:35.764985 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c50c36-87f1-4375-b175-d7385cf170ac None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:35.785583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-96c50c36-87f1-4375-b175-d7385cf170ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:35.785583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:35.785583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:35.785583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:35.785583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:35.785583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:35.785583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:35.785583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:35.785583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:35.785583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:35.785583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:35.785583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:35.785583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:35.785583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:35.785583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:35.787213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:35.787213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:35.787213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:35.787213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:35.787213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:35.787213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:35.787213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:35.787213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:35.787213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:35.787213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:35.787213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:35.787213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:35.787213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:35.787213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:35.787213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:35.787213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:35.787213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:35.789410 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:35.789410 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:35.789410 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:35.789410 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:35.789410 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 318/612] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:35.793507 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea25cc3-940f-4187-8844-a60bf6b90a70 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:35.863256 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052fd4c-4975-42ef-86d5-22355efc6e7e None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:35.863816 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052fd4c-4975-42ef-86d5-22355efc6e7e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:35.864559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052fd4c-4975-42ef-86d5-22355efc6e7e None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:35.871857 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 319/613] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:25:35.877930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db04c0c-27ad-4377-933b-7cd7201d4c75 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:35.878432 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db04c0c-27ad-4377-933b-7cd7201d4c75 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:35.878926 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db04c0c-27ad-4377-933b-7cd7201d4c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:35.925600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5db04c0c-27ad-4377-933b-7cd7201d4c75 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:35.927736 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5db04c0c-27ad-4377-933b-7cd7201d4c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:35.988957 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea25cc3-940f-4187-8844-a60bf6b90a70 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:35.993278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:36.031956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c749f65-9c15-43dd-9f7c-6fe9f6a605d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:36.033129 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c749f65-9c15-43dd-9f7c-6fe9f6a605d3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:36.033129 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c749f65-9c15-43dd-9f7c-6fe9f6a605d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:36.033411 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c749f65-9c15-43dd-9f7c-6fe9f6a605d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:36.034111 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c749f65-9c15-43dd-9f7c-6fe9f6a605d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:36.034424 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c749f65-9c15-43dd-9f7c-6fe9f6a605d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:36.035849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c749f65-9c15-43dd-9f7c-6fe9f6a605d3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:36.052097 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 295/614] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:36.058060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c749f65-9c15-43dd-9f7c-6fe9f6a605d3 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:36.064247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:36.078870 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 320/615] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:36.122472 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db04c0c-27ad-4377-933b-7cd7201d4c75 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:36.175017 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c749f65-9c15-43dd-9f7c-6fe9f6a605d3 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:36.275029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd62e1b-6333-4c9a-a22d-6835734f1a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:36.276337 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd62e1b-6333-4c9a-a22d-6835734f1a50 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:36.276848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd62e1b-6333-4c9a-a22d-6835734f1a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:36.277358 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd62e1b-6333-4c9a-a22d-6835734f1a50 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:36.304578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd62e1b-6333-4c9a-a22d-6835734f1a50 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:36.306984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd62e1b-6333-4c9a-a22d-6835734f1a50 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:36.357335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-1bd62e1b-6333-4c9a-a22d-6835734f1a50 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 08:25:36.357335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:36.357335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:36.357335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:36.357335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:36.357335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:36.357335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:36.357335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:36.357335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:36.357335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:36.357335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:36.357335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:36.357335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:25:36.357335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:25:36.357335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:25:36.357335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:25:36.359855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:36.359855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:36.359855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:25:36.359855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:36.359855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:25:36.359855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:36.359855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:36.359855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:36.359855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:36.359855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:36.359855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:36.359855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:36.359855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:36.359855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:36.359855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:36.359855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:36.359855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:36.359855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:36.361714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:36.361714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:36.361714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:25:36.361714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:25:36.361714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:25:36.361714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:25:36.361714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:25:36.361714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:25:36.361714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 08:25:36.361714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:36.361714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 296/616] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:25:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:36.365374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd62e1b-6333-4c9a-a22d-6835734f1a50 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:36.406506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:36.424820 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db04c0c-27ad-4377-933b-7cd7201d4c75 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:36.482666 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6d36a-e483-4ccd-9b2b-c8d36f2a711c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:36.487318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6d36a-e483-4ccd-9b2b-c8d36f2a711c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:36.487876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6d36a-e483-4ccd-9b2b-c8d36f2a711c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:36.488331 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6d36a-e483-4ccd-9b2b-c8d36f2a711c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:36.518254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b6d36a-e483-4ccd-9b2b-c8d36f2a711c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:36.543254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f4b6d36a-e483-4ccd-9b2b-c8d36f2a711c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:36.543254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:36.543254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:36.543254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:36.543254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:36.543254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:36.543254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:36.543254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:36.543254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:36.543254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:36.543254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:36.543254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:36.543254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:36.543254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:36.543254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:36.543254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:36.544450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:36.544450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:36.544450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:36.544450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:36.544450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:36.544450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:36.544450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:36.544450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:36.544450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:36.544450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:36.544450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:36.544450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:36.544450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:36.544450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:36.544450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:36.544450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:36.545295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:36.545295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:36.545295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:36.545295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:36.545295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:36.545295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:36.545295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:36.545295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:36.545295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:36.545295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:36.545295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:36.545295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:36.559758 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 321/617] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:25:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:36.564927 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6d36a-e483-4ccd-9b2b-c8d36f2a711c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:36.566275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd62e1b-6333-4c9a-a22d-6835734f1a50 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:36.618971 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4fff0-bdaa-4341-ad40-f04749373f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:36.619953 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4fff0-bdaa-4341-ad40-f04749373f74 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:36.623945 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4fff0-bdaa-4341-ad40-f04749373f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:36.624452 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4fff0-bdaa-4341-ad40-f04749373f74 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:36.625517 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae4fff0-bdaa-4341-ad40-f04749373f74 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:36.625990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae4fff0-bdaa-4341-ad40-f04749373f74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:36.627617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae4fff0-bdaa-4341-ad40-f04749373f74 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:36.667393 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 297/618] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Jul 18 08:25:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:36.685431 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4fff0-bdaa-4341-ad40-f04749373f74 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:36.713458 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:36.845890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6d36a-e483-4ccd-9b2b-c8d36f2a711c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:36.949885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:36.973472 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece8bca-4d54-4ee2-8064-c4a7dc8e797e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:36.974744 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece8bca-4d54-4ee2-8064-c4a7dc8e797e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:36.975800 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece8bca-4d54-4ee2-8064-c4a7dc8e797e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:36.976327 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece8bca-4d54-4ee2-8064-c4a7dc8e797e None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:36.977813 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece8bca-4d54-4ee2-8064-c4a7dc8e797e None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:36.978329 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece8bca-4d54-4ee2-8064-c4a7dc8e797e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:36.981040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece8bca-4d54-4ee2-8064-c4a7dc8e797e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:36.997533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 322/619] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:25:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:25:37.005013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece8bca-4d54-4ee2-8064-c4a7dc8e797e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:37.024749 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4fff0-bdaa-4341-ad40-f04749373f74 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:37.142686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42def27-fadb-4bbb-841d-79f326401de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:37.143962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42def27-fadb-4bbb-841d-79f326401de6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:37.144433 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42def27-fadb-4bbb-841d-79f326401de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:37.144920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42def27-fadb-4bbb-841d-79f326401de6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:37.174284 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42def27-fadb-4bbb-841d-79f326401de6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:37.178094 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42def27-fadb-4bbb-841d-79f326401de6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:37.228003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-a42def27-fadb-4bbb-841d-79f326401de6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:37.228003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:37.228003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:37.228003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:37.228003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:37.228003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:37.228003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:37.228003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:37.228003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:37.228003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:37.228003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:37.228003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:37.228003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:37.228003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:37.228003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:37.228985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:37.228985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:37.228985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:37.228985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:37.228985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:37.228985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:37.228985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:37.228985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:37.228985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:37.228985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:37.228985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:37.228985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:37.228985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:37.228985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:37.228985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:37.228985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:37.228985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:37.229881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:37.229881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:37.229881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:37.229881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:37.230674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 298/620] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 545 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:25:37.237435 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42def27-fadb-4bbb-841d-79f326401de6 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:37.296806 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece8bca-4d54-4ee2-8064-c4a7dc8e797e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:37.321918 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:37.344031 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be4546-7431-494e-b03b-2b433a42aa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:37.344863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be4546-7431-494e-b03b-2b433a42aa13 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:37.345150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be4546-7431-494e-b03b-2b433a42aa13 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:37.345432 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be4546-7431-494e-b03b-2b433a42aa13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:37.359452 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be4546-7431-494e-b03b-2b433a42aa13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:37.361133 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be4546-7431-494e-b03b-2b433a42aa13 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:37.383033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-b2be4546-7431-494e-b03b-2b433a42aa13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:37.383033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:37.383033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:37.383033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:37.383033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:37.383033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:37.383033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:37.383033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:37.383033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:37.383033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:37.383033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:37.383033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:37.383033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:37.383033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:37.383033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:37.384058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:37.384058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:37.384058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:37.384058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:37.384058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:37.384058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:37.384058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:37.384058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:37.384058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:37.384058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:37.384058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:37.384058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:37.384058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:37.384058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:37.384058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:37.384058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:37.384058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:37.384966 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:37.384966 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:37.384966 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:37.384966 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:37.384966 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 323/621] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:25:37.386652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3bd4ff-38ba-4454-b37e-8799d4b26383 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:37.386652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3bd4ff-38ba-4454-b37e-8799d4b26383 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:37.386982 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3bd4ff-38ba-4454-b37e-8799d4b26383 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:37.392413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 324/622] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:37.393563 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3bd4ff-38ba-4454-b37e-8799d4b26383 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:37.440900 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42def27-fadb-4bbb-841d-79f326401de6 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:37.502317 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:37.509704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b031e-11fe-49fa-8e31-72587bee0457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:37.510452 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b031e-11fe-49fa-8e31-72587bee0457 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:37.512832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b031e-11fe-49fa-8e31-72587bee0457 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:37.512832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b031e-11fe-49fa-8e31-72587bee0457 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:37.512832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b031e-11fe-49fa-8e31-72587bee0457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:37.512832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b031e-11fe-49fa-8e31-72587bee0457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:37.513586 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b031e-11fe-49fa-8e31-72587bee0457 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:37.531924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 299/623] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:25:37.531924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d9505-3dca-40d3-a894-19c161831a08 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:37.531924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d9505-3dca-40d3-a894-19c161831a08 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:37.531924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d9505-3dca-40d3-a894-19c161831a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:37.550303 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-272d9505-3dca-40d3-a894-19c161831a08 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:37.551483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-272d9505-3dca-40d3-a894-19c161831a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:37.564068 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3bd4ff-38ba-4454-b37e-8799d4b26383 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:37.613399 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713d2d8-64e5-4abd-8745-81f0b076695f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:37.614079 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713d2d8-64e5-4abd-8745-81f0b076695f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:37.614620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713d2d8-64e5-4abd-8745-81f0b076695f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:37.614901 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713d2d8-64e5-4abd-8745-81f0b076695f None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:37.616953 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713d2d8-64e5-4abd-8745-81f0b076695f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:37.617271 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713d2d8-64e5-4abd-8745-81f0b076695f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:37.618683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713d2d8-64e5-4abd-8745-81f0b076695f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:37.635323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 325/624] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:37.635323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c3245-7aa1-4153-990c-b7a570f64b5e None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:37.635323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c3245-7aa1-4153-990c-b7a570f64b5e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:37.635323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c3245-7aa1-4153-990c-b7a570f64b5e None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:37.635323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 326/625] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:37.636856 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c3245-7aa1-4153-990c-b7a570f64b5e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:37.647803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 300/626] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:37.647803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d9505-3dca-40d3-a894-19c161831a08 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:37.676066 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:37.784009 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c3245-7aa1-4153-990c-b7a570f64b5e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:37.787107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d9505-3dca-40d3-a894-19c161831a08 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:37.816338 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:37.829813 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41796f50-50f3-4731-86d3-96bd72c24806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:37.830631 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41796f50-50f3-4731-86d3-96bd72c24806 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:37.830914 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41796f50-50f3-4731-86d3-96bd72c24806 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:37.831234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41796f50-50f3-4731-86d3-96bd72c24806 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:37.834677 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709e829-9430-499f-bfdc-9c3e33cd71e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:37.835697 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709e829-9430-499f-bfdc-9c3e33cd71e5 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:37.835944 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709e829-9430-499f-bfdc-9c3e33cd71e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:37.836222 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709e829-9430-499f-bfdc-9c3e33cd71e5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:37.845144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f709e829-9430-499f-bfdc-9c3e33cd71e5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:37.846622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f709e829-9430-499f-bfdc-9c3e33cd71e5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:37.847841 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41796f50-50f3-4731-86d3-96bd72c24806 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:37.852065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41796f50-50f3-4731-86d3-96bd72c24806 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:37.864546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-f709e829-9430-499f-bfdc-9c3e33cd71e5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 08:25:37.864546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:37.864546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:37.864546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:37.864546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:37.864546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:37.864546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:37.864546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:37.864546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:37.864546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:37.864546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:37.864546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:37.864546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:25:37.864546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:25:37.864546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:25:37.864546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:25:37.865637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:37.865637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:37.865637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:25:37.865637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:37.865637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:25:37.865637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:37.865637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:37.865637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:37.865637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:37.865637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:37.865637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:37.865637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:37.865637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:37.865637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:37.865637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:37.865637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:37.865637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:37.865637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:37.867346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:37.867346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:37.867346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:25:37.867346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:25:37.867346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:25:37.867346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:25:37.867346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:25:37.867346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:25:37.867346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 08:25:37.867346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:37.868531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-41796f50-50f3-4731-86d3-96bd72c24806 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:37.868531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:37.868531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:37.868531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:37.868531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:37.868531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:37.868531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:37.868531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:37.868531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:37.868531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:37.868531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:37.868531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:37.868531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:25:37.868531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:25:37.868531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:25:37.877721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:25:37.877721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:37.877721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:37.877721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:37.877721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:37.877721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:37.877721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:37.877721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:37.877721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:37.877721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:37.877721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:37.877721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:37.877721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:37.877721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:37.877721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:37.877721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:37.877721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:37.882567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:25:37.882567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:25:37.882567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:25:37.882567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:25:37.882567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:25:37.882567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:25:37.882567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:37.882567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:37.882567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 327/627] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:37.882567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8090085-6e1a-4831-80b8-4627b6e4b031 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:37.885351 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 301/628] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 18 08:25:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:37.885497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8090085-6e1a-4831-80b8-4627b6e4b031 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:37.885497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8090085-6e1a-4831-80b8-4627b6e4b031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:37.885832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709e829-9430-499f-bfdc-9c3e33cd71e5 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:37.893971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f8090085-6e1a-4831-80b8-4627b6e4b031 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:37.895225 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f8090085-6e1a-4831-80b8-4627b6e4b031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:37.979875 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 328/629] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:37.981741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8090085-6e1a-4831-80b8-4627b6e4b031 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:38.030369 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709e829-9430-499f-bfdc-9c3e33cd71e5 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:38.037113 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:38.074737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e009e-6f04-49cf-9b6b-aa6a37f7a78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:38.075657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e009e-6f04-49cf-9b6b-aa6a37f7a78f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:38.075971 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e009e-6f04-49cf-9b6b-aa6a37f7a78f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:38.076253 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e009e-6f04-49cf-9b6b-aa6a37f7a78f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:38.090607 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e009e-6f04-49cf-9b6b-aa6a37f7a78f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:38.099016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-d11e009e-6f04-49cf-9b6b-aa6a37f7a78f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:38.099016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:38.099016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:38.099016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:38.099016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:38.099016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:38.099016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:38.099016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:38.099016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:38.099016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:38.099016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:38.099016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:38.099016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:38.099016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:38.099016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:38.099016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:38.100121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:38.100121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:38.100121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:38.100121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:38.100121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:38.100121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:38.100121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:38.100121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:38.100121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:38.100121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:38.100121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:38.100121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:38.100121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:38.100121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:38.100121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:38.100121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:38.100978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:38.100978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:38.100978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:38.100978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:38.100978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:38.100978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:38.100978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:38.100978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:38.100978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:38.100978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:38.100978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:38.100978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:38.102094 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 302/630] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:25:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:38.103072 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e009e-6f04-49cf-9b6b-aa6a37f7a78f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:38.158480 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8090085-6e1a-4831-80b8-4627b6e4b031 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:38.166284 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:38.211134 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9a9ac-1ad2-4719-b903-3f2b4e269636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:38.211848 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9a9ac-1ad2-4719-b903-3f2b4e269636 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:38.212128 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9a9ac-1ad2-4719-b903-3f2b4e269636 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:38.212402 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9a9ac-1ad2-4719-b903-3f2b4e269636 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:38.213220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9a9ac-1ad2-4719-b903-3f2b4e269636 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:38.213531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9a9ac-1ad2-4719-b903-3f2b4e269636 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:38.214993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9a9ac-1ad2-4719-b903-3f2b4e269636 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:38.228178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 329/631] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:25:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:38.229047 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9a9ac-1ad2-4719-b903-3f2b4e269636 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:38.256129 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e009e-6f04-49cf-9b6b-aa6a37f7a78f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:38.302198 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f7f90-8272-4e2c-a696-e08501ef74f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:38.303783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f7f90-8272-4e2c-a696-e08501ef74f8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:38.304254 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f7f90-8272-4e2c-a696-e08501ef74f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:38.304734 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f7f90-8272-4e2c-a696-e08501ef74f8 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:38.305771 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f7f90-8272-4e2c-a696-e08501ef74f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:38.306265 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f7f90-8272-4e2c-a696-e08501ef74f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:38.307910 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f7f90-8272-4e2c-a696-e08501ef74f8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:38.330972 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 303/632] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 18 08:25:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:38.332001 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f7f90-8272-4e2c-a696-e08501ef74f8 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:38.363941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:38.396529 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9a9ac-1ad2-4719-b903-3f2b4e269636 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:38.447955 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a5f88-3e04-440d-a4f5-995fade53d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:38.449307 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a5f88-3e04-440d-a4f5-995fade53d5e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:38.449806 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a5f88-3e04-440d-a4f5-995fade53d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:38.450277 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a5f88-3e04-440d-a4f5-995fade53d5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:38.463603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:38.468937 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f7f90-8272-4e2c-a696-e08501ef74f8 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:38.470582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a5f88-3e04-440d-a4f5-995fade53d5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:38.487345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-112a5f88-3e04-440d-a4f5-995fade53d5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:38.487345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:38.487345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:38.487345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:38.487345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:38.487345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:38.487345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:38.487345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:38.487345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:38.487345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:38.487345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:38.487345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:38.487345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:38.487345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:38.487345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:38.487345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:38.488422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:38.488422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:38.488422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:38.488422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:38.488422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:38.488422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:38.488422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:38.488422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:38.488422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:38.488422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:38.488422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:38.488422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:38.488422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:38.488422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:38.488422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:38.488422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:38.489533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:38.489533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:38.489533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:38.489533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:38.489533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:38.489533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:38.489533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:38.489533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:38.489533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:38.489533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:38.489533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:38.489533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:38.490968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 330/633] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:25:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:38.492320 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a5f88-3e04-440d-a4f5-995fade53d5e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:38.520561 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31438950-4262-4f20-b5ab-0e69e4858e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:38.521138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31438950-4262-4f20-b5ab-0e69e4858e20 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:38.521425 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31438950-4262-4f20-b5ab-0e69e4858e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:38.521702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31438950-4262-4f20-b5ab-0e69e4858e20 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:38.523236 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31438950-4262-4f20-b5ab-0e69e4858e20 None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:38.523544 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31438950-4262-4f20-b5ab-0e69e4858e20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:38.524997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31438950-4262-4f20-b5ab-0e69e4858e20 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:38.538500 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 304/634] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:25:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:38.540063 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31438950-4262-4f20-b5ab-0e69e4858e20 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:38.618060 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a5f88-3e04-440d-a4f5-995fade53d5e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:38.665344 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53709d-2f7b-422e-822e-49527652f987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:38.666890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53709d-2f7b-422e-822e-49527652f987 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:38.667231 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53709d-2f7b-422e-822e-49527652f987 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:38.667735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53709d-2f7b-422e-822e-49527652f987 None admin] PATH_INFO: `/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/09a5c2e804504038817cad76544dce5f/roles/9fc4c3dcb90945ebb4eb5566e62e7833` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:38.668283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31438950-4262-4f20-b5ab-0e69e4858e20 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:38.687484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:38.693896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53709d-2f7b-422e-822e-49527652f987 None admin] RBAC: Authorizing `identity:create_grant(project_id=35c846931fd443fbb83a6fbe0aeaa866, user_id=09a5c2e804504038817cad76544dce5f, role_id=9fc4c3dcb90945ebb4eb5566e62e7833)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:38.696657 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53709d-2f7b-422e-822e-49527652f987 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:38.704620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5dd10-eefa-44e4-8e32-afea36a70601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:38.705758 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5dd10-eefa-44e4-8e32-afea36a70601 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:38.706221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5dd10-eefa-44e4-8e32-afea36a70601 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:38.706663 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5dd10-eefa-44e4-8e32-afea36a70601 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:38.745663 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:38.790268 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 331/635] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:25:38 2022] PUT /identity/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/09a5c2e804504038817cad76544dce5f/roles/9fc4c3dcb90945ebb4eb5566e62e7833 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:25:38.791378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53709d-2f7b-422e-822e-49527652f987 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:38.798977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5dd10-eefa-44e4-8e32-afea36a70601 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:38.800534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5dd10-eefa-44e4-8e32-afea36a70601 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:38.968919 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53709d-2f7b-422e-822e-49527652f987 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:38.980394 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 305/636] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:25:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:25:38.982442 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7fddc-01bf-4738-9254-779aac9c8c24 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:38.982749 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7fddc-01bf-4738-9254-779aac9c8c24 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:38.983003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7fddc-01bf-4738-9254-779aac9c8c24 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:38.985418 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 306/637] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:38.986305 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7fddc-01bf-4738-9254-779aac9c8c24 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:39.027109 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a203c05-8093-415e-bf03-d8daee0f71c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:39.028293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a203c05-8093-415e-bf03-d8daee0f71c2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:39.029051 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a203c05-8093-415e-bf03-d8daee0f71c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:39.030713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a203c05-8093-415e-bf03-d8daee0f71c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:39.032218 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a203c05-8093-415e-bf03-d8daee0f71c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:39.032783 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a203c05-8093-415e-bf03-d8daee0f71c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:39.034291 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a203c05-8093-415e-bf03-d8daee0f71c2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:39.047895 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 332/638] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:25:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:39.050112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a203c05-8093-415e-bf03-d8daee0f71c2 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:39.059149 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:39.107752 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7fddc-01bf-4738-9254-779aac9c8c24 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:39.139921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:39.162340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d29942-826c-43bb-b9bf-7407bb17371d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:39.165169 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d29942-826c-43bb-b9bf-7407bb17371d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:39.165812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d29942-826c-43bb-b9bf-7407bb17371d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:39.166186 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d29942-826c-43bb-b9bf-7407bb17371d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:39.185133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d29942-826c-43bb-b9bf-7407bb17371d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:39.186610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d29942-826c-43bb-b9bf-7407bb17371d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:39.202141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a203c05-8093-415e-bf03-d8daee0f71c2 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:39.205629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-f5d29942-826c-43bb-b9bf-7407bb17371d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:39.205629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:39.205629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:39.205629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:39.205629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:39.205629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:39.205629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:39.205629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:39.205629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:39.205629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:39.205629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:39.205629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:39.205629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:39.205629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:39.205629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:39.207240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:39.207240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:39.207240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:39.207240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:39.207240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:39.207240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:39.207240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:39.207240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:39.207240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:39.207240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:39.207240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:39.207240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:39.207240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:39.207240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:39.207240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:39.207240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:39.207240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:39.209329 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:39.209329 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:39.209329 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:39.209329 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:39.209329 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 307/639] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:39.209329 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a70e1-a4f8-48d7-a1d5-ee9c5358fc1c None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:39.209329 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a70e1-a4f8-48d7-a1d5-ee9c5358fc1c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:39.209329 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a70e1-a4f8-48d7-a1d5-ee9c5358fc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:39.234275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c54a70e1-a4f8-48d7-a1d5-ee9c5358fc1c None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:39.235843 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c54a70e1-a4f8-48d7-a1d5-ee9c5358fc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:39.270710 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c366d-99d3-4f71-a63c-79ded6d9d46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:39.273991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c366d-99d3-4f71-a63c-79ded6d9d46d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:39.274546 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c366d-99d3-4f71-a63c-79ded6d9d46d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:39.275005 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c366d-99d3-4f71-a63c-79ded6d9d46d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:39.291417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c366d-99d3-4f71-a63c-79ded6d9d46d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:39.293044 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c366d-99d3-4f71-a63c-79ded6d9d46d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:39.315535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-179c366d-99d3-4f71-a63c-79ded6d9d46d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:39.315535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:39.315535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:39.315535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:39.315535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:39.315535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:39.315535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:39.315535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:39.315535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:39.315535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:39.315535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:39.315535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:39.315535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:39.315535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:39.315535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:39.325006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:39.325006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:39.325006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:39.325006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:39.325006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:39.325006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:39.325006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:39.325006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:39.325006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:39.325006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:39.325006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:39.325006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:39.325006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:39.325006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:39.325006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:39.325006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:39.325006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:39.327049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:39.327049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:39.327049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:39.327049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:39.327049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 333/640] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:39.327049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c366d-99d3-4f71-a63c-79ded6d9d46d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:39.366721 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 308/641] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:39.368829 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a70e1-a4f8-48d7-a1d5-ee9c5358fc1c None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:39.448966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:39.511911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c366d-99d3-4f71-a63c-79ded6d9d46d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:39.515174 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:39.560655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a70e1-a4f8-48d7-a1d5-ee9c5358fc1c None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:39.577819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b92d4-e69a-47dc-8a4b-2f83456e7d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:39.579098 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b92d4-e69a-47dc-8a4b-2f83456e7d7c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:39.579590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b92d4-e69a-47dc-8a4b-2f83456e7d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:39.580113 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b92d4-e69a-47dc-8a4b-2f83456e7d7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:39.601407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b92d4-e69a-47dc-8a4b-2f83456e7d7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:39.604108 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b92d4-e69a-47dc-8a4b-2f83456e7d7c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:39.622822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a48cbe-39eb-48d0-8d3b-b117b29f30af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:39.623801 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a48cbe-39eb-48d0-8d3b-b117b29f30af None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:39.624092 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a48cbe-39eb-48d0-8d3b-b117b29f30af None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:39.624375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a48cbe-39eb-48d0-8d3b-b117b29f30af None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:39.625122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a48cbe-39eb-48d0-8d3b-b117b29f30af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:39.625429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a48cbe-39eb-48d0-8d3b-b117b29f30af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:39.626823 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a48cbe-39eb-48d0-8d3b-b117b29f30af None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:39.638118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-0b6b92d4-e69a-47dc-8a4b-2f83456e7d7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:39.638118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:39.638118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:39.638118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:39.638118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:39.638118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:39.638118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:39.638118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:39.638118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:39.638118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:39.638118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:39.638118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:39.638118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:39.638118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:39.638118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:39.640254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:39.640254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:39.640254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:39.640254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:39.640254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:39.640254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:39.640254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:39.640254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:39.640254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:39.640254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:39.640254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:39.640254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:39.640254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:39.640254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:39.640254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:39.640254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:39.640254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:39.642094 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:39.642094 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:39.642094 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:39.642094 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:39.642094 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 334/642] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:39.642094 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b92d4-e69a-47dc-8a4b-2f83456e7d7c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:39.656301 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 309/643] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:39.656301 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a48cbe-39eb-48d0-8d3b-b117b29f30af None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:39.827043 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b92d4-e69a-47dc-8a4b-2f83456e7d7c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:39.832789 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a48cbe-39eb-48d0-8d3b-b117b29f30af None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:39.848878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:39.884913 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131bbd51-d564-40a1-9c4c-ab4671a86787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:39.885734 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bbd51-d564-40a1-9c4c-ab4671a86787 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:39.886005 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bbd51-d564-40a1-9c4c-ab4671a86787 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:39.886337 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bbd51-d564-40a1-9c4c-ab4671a86787 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:39.887190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e1f76-7fe5-49b6-a810-86c042f07c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:39.888047 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e1f76-7fe5-49b6-a810-86c042f07c06 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:39.888511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e1f76-7fe5-49b6-a810-86c042f07c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:39.888930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e1f76-7fe5-49b6-a810-86c042f07c06 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:39.889836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e1f76-7fe5-49b6-a810-86c042f07c06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:39.890334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e1f76-7fe5-49b6-a810-86c042f07c06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:39.891875 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e1f76-7fe5-49b6-a810-86c042f07c06 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:39.902679 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131bbd51-d564-40a1-9c4c-ab4671a86787 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:39.907004 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 335/644] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:39.908656 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e1f76-7fe5-49b6-a810-86c042f07c06 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:39.912275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-131bbd51-d564-40a1-9c4c-ab4671a86787 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:39.912275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:39.912275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:39.912275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:39.912275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:39.912275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:39.912275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:39.912275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:39.912275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:39.912275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:39.912275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:39.912275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:39.912275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:39.912275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:39.912275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:39.912275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:39.913932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:39.913932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:39.913932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:39.913932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:39.913932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:39.913932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:39.913932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:39.913932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:39.913932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:39.913932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:39.913932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:39.913932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:39.913932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:39.913932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:39.913932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:39.913932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:39.914839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:39.914839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:39.914839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:39.914839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:39.914839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:39.914839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:39.914839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:39.914839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:39.914839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:39.914839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:39.914839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:39.914839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:39.916134 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 310/645] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:25:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:39.917077 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131bbd51-d564-40a1-9c4c-ab4671a86787 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:39.946249 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:40.075754 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e1f76-7fe5-49b6-a810-86c042f07c06 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:40.077783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131bbd51-d564-40a1-9c4c-ab4671a86787 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:40.118351 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c2c6b-33ad-4a32-b18c-611a58bd178a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:40.119409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c2c6b-33ad-4a32-b18c-611a58bd178a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:40.119975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c2c6b-33ad-4a32-b18c-611a58bd178a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:40.120474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c2c6b-33ad-4a32-b18c-611a58bd178a None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:40.121941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c2c6b-33ad-4a32-b18c-611a58bd178a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:40.122493 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c2c6b-33ad-4a32-b18c-611a58bd178a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:40.125178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c2c6b-33ad-4a32-b18c-611a58bd178a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:40.137681 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef05392-88ed-48eb-982b-a648867a6bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:40.138506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef05392-88ed-48eb-982b-a648867a6bbf None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:40.138793 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef05392-88ed-48eb-982b-a648867a6bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:40.139071 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef05392-88ed-48eb-982b-a648867a6bbf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:40.153657 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 336/646] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:40.153657 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c2c6b-33ad-4a32-b18c-611a58bd178a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:40.155809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef05392-88ed-48eb-982b-a648867a6bbf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:40.158277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef05392-88ed-48eb-982b-a648867a6bbf None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:40.175850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-1ef05392-88ed-48eb-982b-a648867a6bbf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 08:25:40.175850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:40.175850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:40.175850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:40.175850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:40.175850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:40.175850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:40.175850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:40.175850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:40.175850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:40.175850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:40.175850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:40.175850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:25:40.175850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:25:40.175850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:25:40.175850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:25:40.176873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:40.176873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:40.176873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:25:40.176873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:40.176873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:25:40.176873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:40.176873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:40.176873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:40.176873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:40.176873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:40.176873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:40.176873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:40.176873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:40.176873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:40.176873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:40.176873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:40.176873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:40.176873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:40.177824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:40.177824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:40.177824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:25:40.177824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:25:40.177824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:25:40.177824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:25:40.177824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:25:40.177824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:25:40.177824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 08:25:40.177824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:40.180139 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 311/647] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:25:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:40.181869 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef05392-88ed-48eb-982b-a648867a6bbf None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:40.229099 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:40.258305 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:40.294690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c2c6b-33ad-4a32-b18c-611a58bd178a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:40.333082 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d7a5f-4a54-4d16-a0bc-f7c5f6f01432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:40.333897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d7a5f-4a54-4d16-a0bc-f7c5f6f01432 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:40.334187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d7a5f-4a54-4d16-a0bc-f7c5f6f01432 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:40.334504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d7a5f-4a54-4d16-a0bc-f7c5f6f01432 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:40.342136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d7a5f-4a54-4d16-a0bc-f7c5f6f01432 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:40.343635 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d7a5f-4a54-4d16-a0bc-f7c5f6f01432 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:40.352554 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef05392-88ed-48eb-982b-a648867a6bbf None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:40.358733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-4e3d7a5f-4a54-4d16-a0bc-f7c5f6f01432 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:40.358733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:40.358733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:40.358733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:40.358733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:40.358733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:40.358733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:40.358733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:40.358733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:40.358733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:40.358733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:40.358733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:40.358733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:25:40.358733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:25:40.358733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:25:40.359662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:25:40.359662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:40.359662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:40.359662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:40.359662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:40.359662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:40.359662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:40.359662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:40.359662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:40.359662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:40.359662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:40.359662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:40.359662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:40.359662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:40.359662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:40.359662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:40.359662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:40.360580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:25:40.360580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:25:40.360580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:25:40.360580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:25:40.360580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:25:40.360580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:25:40.360580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:40.360580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:40.360580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 337/648] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:40.361619 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d7a5f-4a54-4d16-a0bc-f7c5f6f01432 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:40.408594 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ef7c9-f087-4fd4-b300-c0a020b4b010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:40.409690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ef7c9-f087-4fd4-b300-c0a020b4b010 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:40.410714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ef7c9-f087-4fd4-b300-c0a020b4b010 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:40.411222 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ef7c9-f087-4fd4-b300-c0a020b4b010 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:40.414318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ef7c9-f087-4fd4-b300-c0a020b4b010 None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:40.414918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ef7c9-f087-4fd4-b300-c0a020b4b010 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:40.417507 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ef7c9-f087-4fd4-b300-c0a020b4b010 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:40.432061 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 312/649] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:25:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:40.433547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ef7c9-f087-4fd4-b300-c0a020b4b010 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:40.517603 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d7a5f-4a54-4d16-a0bc-f7c5f6f01432 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:40.571974 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4b01b-7558-401f-91dd-5842801878ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:40.573704 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4b01b-7558-401f-91dd-5842801878ca None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:40.574476 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4b01b-7558-401f-91dd-5842801878ca None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:40.575226 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4b01b-7558-401f-91dd-5842801878ca None admin] PATH_INFO: `/v3/users/glance` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:40.580420 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:40.581436 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:40.591807 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f4b01b-7558-401f-91dd-5842801878ca None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:40.594436 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f4b01b-7558-401f-91dd-5842801878ca None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:40.610357 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ef7c9-f087-4fd4-b300-c0a020b4b010 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:40.620678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-56f4b01b-7558-401f-91dd-5842801878ca None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 08:25:40.620678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:40.620678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:40.620678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:40.620678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:40.620678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:40.620678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:40.620678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:40.620678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:40.620678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:40.620678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:40.620678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:40.620678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:25:40.620678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:25:40.620678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:25:40.620678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:25:40.628239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:40.628239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:40.628239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:25:40.628239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:40.628239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:25:40.628239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:40.628239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:40.628239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:40.628239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:40.628239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:40.628239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:40.628239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:40.628239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:40.628239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:40.628239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:40.628239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:40.628239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:40.628239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:40.630089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:40.630089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:40.630089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:25:40.630089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:25:40.630089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:25:40.630089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:25:40.630089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:25:40.630089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:25:40.630089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 08:25:40.630089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:40.630089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 338/650] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:25:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:40.630089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4b01b-7558-401f-91dd-5842801878ca None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:40.689357 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4dc43-0014-4422-9a29-c7a7d2e29e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:40.690647 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4dc43-0014-4422-9a29-c7a7d2e29e5f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:40.691153 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4dc43-0014-4422-9a29-c7a7d2e29e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:40.691641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4dc43-0014-4422-9a29-c7a7d2e29e5f None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:40.693701 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee4dc43-0014-4422-9a29-c7a7d2e29e5f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:40.694917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee4dc43-0014-4422-9a29-c7a7d2e29e5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:40.698154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee4dc43-0014-4422-9a29-c7a7d2e29e5f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:40.733358 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 313/651] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Jul 18 08:25:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:40.734218 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4dc43-0014-4422-9a29-c7a7d2e29e5f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:40.768331 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4b01b-7558-401f-91dd-5842801878ca None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:40.812649 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceeabec-17bb-4c5c-a3d1-efd3baa9c09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:40.813494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceeabec-17bb-4c5c-a3d1-efd3baa9c09f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:40.813929 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceeabec-17bb-4c5c-a3d1-efd3baa9c09f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:40.814346 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceeabec-17bb-4c5c-a3d1-efd3baa9c09f None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:40.815293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceeabec-17bb-4c5c-a3d1-efd3baa9c09f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:40.815771 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceeabec-17bb-4c5c-a3d1-efd3baa9c09f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:40.817773 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceeabec-17bb-4c5c-a3d1-efd3baa9c09f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:40.839271 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 339/652] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:25:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:40.841304 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceeabec-17bb-4c5c-a3d1-efd3baa9c09f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:40.912491 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4dc43-0014-4422-9a29-c7a7d2e29e5f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:40.958450 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84a9ea-230c-4ed3-aa8d-814ca8f894fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:40.959943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84a9ea-230c-4ed3-aa8d-814ca8f894fa None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:40.960409 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84a9ea-230c-4ed3-aa8d-814ca8f894fa None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:40.960850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84a9ea-230c-4ed3-aa8d-814ca8f894fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:40.973896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84a9ea-230c-4ed3-aa8d-814ca8f894fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:40.976187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84a9ea-230c-4ed3-aa8d-814ca8f894fa None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:40.978499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:40.979051 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:40.999365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-2f84a9ea-230c-4ed3-aa8d-814ca8f894fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:40.999365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:40.999365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:40.999365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:40.999365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:40.999365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:40.999365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:40.999365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:40.999365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:40.999365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:40.999365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:40.999365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:40.999365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:40.999365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:40.999365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:41.000370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:41.000370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:41.000370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:41.000370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:41.000370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:41.000370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:41.000370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:41.000370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:41.000370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:41.000370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:41.000370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:41.000370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:41.000370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:41.000370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:41.000370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:41.000370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:41.000370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:41.001264 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:41.001264 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:41.001264 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:41.001264 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:41.001264 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 314/653] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:41.001977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84a9ea-230c-4ed3-aa8d-814ca8f894fa None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:41.058010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceeabec-17bb-4c5c-a3d1-efd3baa9c09f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:41.118202 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad3324-316b-4fde-b928-520dd43c4cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:41.119335 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad3324-316b-4fde-b928-520dd43c4cda None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:41.120709 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad3324-316b-4fde-b928-520dd43c4cda None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:41.121242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad3324-316b-4fde-b928-520dd43c4cda None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:41.123356 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad3324-316b-4fde-b928-520dd43c4cda None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:41.123929 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad3324-316b-4fde-b928-520dd43c4cda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:41.126968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad3324-316b-4fde-b928-520dd43c4cda None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:41.169122 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 340/654] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Jul 18 08:25:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:41.169122 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad3324-316b-4fde-b928-520dd43c4cda None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:41.173079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84a9ea-230c-4ed3-aa8d-814ca8f894fa None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:41.217772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37547c7-d6f2-4de8-b04b-9b881c940089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:41.218452 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37547c7-d6f2-4de8-b04b-9b881c940089 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:41.218833 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37547c7-d6f2-4de8-b04b-9b881c940089 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:41.219119 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37547c7-d6f2-4de8-b04b-9b881c940089 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:41.300597 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:41.312967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37547c7-d6f2-4de8-b04b-9b881c940089 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:41.315855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37547c7-d6f2-4de8-b04b-9b881c940089 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:41.338376 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad3324-316b-4fde-b928-520dd43c4cda None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:41.354375 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:41.375775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab014d83-48af-4f54-b919-718253ad308e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:41.376443 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab014d83-48af-4f54-b919-718253ad308e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:41.376726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab014d83-48af-4f54-b919-718253ad308e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:41.377000 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab014d83-48af-4f54-b919-718253ad308e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:41.377843 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab014d83-48af-4f54-b919-718253ad308e None admin] RBAC: Adding query filter params (role.id=9fc4c3dcb90945ebb4eb5566e62e7833, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=84cba474fc7c470ea119de697b9d7ef9) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:41.378153 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab014d83-48af-4f54-b919-718253ad308e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:41.379612 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab014d83-48af-4f54-b919-718253ad308e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:41.387588 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 341/655] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:25:41 2022] GET /identity/v3/role_assignments?user.id=84cba474fc7c470ea119de697b9d7ef9&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=9fc4c3dcb90945ebb4eb5566e62e7833 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:41.389274 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab014d83-48af-4f54-b919-718253ad308e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:41.450764 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 315/656] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:25:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:25:41.452394 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37547c7-d6f2-4de8-b04b-9b881c940089 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:41.575232 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab014d83-48af-4f54-b919-718253ad308e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:41.607375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:41.617379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37547c7-d6f2-4de8-b04b-9b881c940089 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:41.619034 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653554eb-dfec-46c7-a854-23e9ed3d5b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:41.619746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653554eb-dfec-46c7-a854-23e9ed3d5b35 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:41.620032 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653554eb-dfec-46c7-a854-23e9ed3d5b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:41.621157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653554eb-dfec-46c7-a854-23e9ed3d5b35 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:41.621924 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653554eb-dfec-46c7-a854-23e9ed3d5b35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:41.622232 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653554eb-dfec-46c7-a854-23e9ed3d5b35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:41.623640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653554eb-dfec-46c7-a854-23e9ed3d5b35 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:41.650754 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 342/657] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:41.653346 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b5dfd-ecbe-4479-9631-30bdcd4bd5ca None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:41.654067 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b5dfd-ecbe-4479-9631-30bdcd4bd5ca None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:41.654718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b5dfd-ecbe-4479-9631-30bdcd4bd5ca None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:41.657267 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 343/658] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:41.660593 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b5dfd-ecbe-4479-9631-30bdcd4bd5ca None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:41.674906 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95262a-b0c1-45f6-b08a-56203d8f6dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:41.676092 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95262a-b0c1-45f6-b08a-56203d8f6dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:41.676383 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95262a-b0c1-45f6-b08a-56203d8f6dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:41.676659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95262a-b0c1-45f6-b08a-56203d8f6dc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:41.713375 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:41.768477 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95262a-b0c1-45f6-b08a-56203d8f6dc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:41.770352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95262a-b0c1-45f6-b08a-56203d8f6dc3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:41.790661 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b5dfd-ecbe-4479-9631-30bdcd4bd5ca None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:41.840084 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061086f-82cd-481c-bfbc-29ec2c764e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:41.840831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061086f-82cd-481c-bfbc-29ec2c764e46 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:41.841120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061086f-82cd-481c-bfbc-29ec2c764e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:41.841395 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061086f-82cd-481c-bfbc-29ec2c764e46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:41.855427 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061086f-82cd-481c-bfbc-29ec2c764e46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:41.856952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061086f-82cd-481c-bfbc-29ec2c764e46 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:41.877419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-3061086f-82cd-481c-bfbc-29ec2c764e46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:41.877419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:41.877419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:41.877419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:41.877419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:41.877419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:41.877419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:41.877419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:41.877419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:41.877419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:41.877419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:41.877419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:41.877419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:41.877419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:41.877419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:41.878695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:41.878695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:41.878695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:41.878695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:41.878695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:41.878695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:41.878695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:41.878695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:41.878695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:41.878695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:41.878695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:41.878695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:41.878695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:41.878695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:41.878695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:41.878695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:41.878695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:41.880100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:41.880100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:41.880100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:41.880100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:41.884072 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 344/659] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:41.885734 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061086f-82cd-481c-bfbc-29ec2c764e46 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:41.920838 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:41.934911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 316/660] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:25:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:25:41.940465 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774b56c-2b4c-4b6d-bc28-2c089f7fad90 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:41.940465 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774b56c-2b4c-4b6d-bc28-2c089f7fad90 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:41.940465 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774b56c-2b4c-4b6d-bc28-2c089f7fad90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:41.958702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-4774b56c-2b4c-4b6d-bc28-2c089f7fad90 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:41.961642 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4774b56c-2b4c-4b6d-bc28-2c089f7fad90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:42.020336 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:42.032015 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061086f-82cd-481c-bfbc-29ec2c764e46 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:42.066602 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 317/661] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:42.067209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774b56c-2b4c-4b6d-bc28-2c089f7fad90 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:42.093861 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc85afd-ad4e-4327-9422-23ffa9582c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:42.095748 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc85afd-ad4e-4327-9422-23ffa9582c05 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:42.096087 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc85afd-ad4e-4327-9422-23ffa9582c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:42.096376 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc85afd-ad4e-4327-9422-23ffa9582c05 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:42.109190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc85afd-ad4e-4327-9422-23ffa9582c05 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:42.110671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc85afd-ad4e-4327-9422-23ffa9582c05 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:42.135112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-4fc85afd-ad4e-4327-9422-23ffa9582c05 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 08:25:42.135112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:42.135112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:42.135112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:42.135112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:42.135112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:42.135112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:42.135112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:42.135112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:42.135112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:42.135112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:42.135112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:42.135112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:25:42.135112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:25:42.135112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:25:42.135112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:25:42.138139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:42.138139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:42.138139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:25:42.138139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:42.138139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:25:42.138139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:42.138139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:42.138139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:42.138139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:42.138139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:42.138139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:42.138139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:42.138139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:42.138139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:42.138139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:42.138139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:42.138139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:42.138139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:42.139716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:42.139716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:42.139716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:25:42.139716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:25:42.139716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:25:42.139716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:25:42.139716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:25:42.139716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:25:42.139716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 08:25:42.139716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:42.142167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 345/662] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:25:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:42.142683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc85afd-ad4e-4327-9422-23ffa9582c05 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:42.222786 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774b56c-2b4c-4b6d-bc28-2c089f7fad90 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:42.264348 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:42.276843 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc85afd-ad4e-4327-9422-23ffa9582c05 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:42.281837 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421ddd9-b008-4c61-ab35-0e7a540bf872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:42.283079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421ddd9-b008-4c61-ab35-0e7a540bf872 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:42.283504 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421ddd9-b008-4c61-ab35-0e7a540bf872 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:42.283890 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421ddd9-b008-4c61-ab35-0e7a540bf872 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:42.284808 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421ddd9-b008-4c61-ab35-0e7a540bf872 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:42.285127 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421ddd9-b008-4c61-ab35-0e7a540bf872 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:42.286777 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421ddd9-b008-4c61-ab35-0e7a540bf872 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:42.311151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 318/663] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:42.312662 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421ddd9-b008-4c61-ab35-0e7a540bf872 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:42.321717 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d28620-8152-4f9e-adf1-530299244421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:42.322734 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d28620-8152-4f9e-adf1-530299244421 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:42.323159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d28620-8152-4f9e-adf1-530299244421 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:42.323569 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d28620-8152-4f9e-adf1-530299244421 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:42.338342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d28620-8152-4f9e-adf1-530299244421 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:42.340295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d28620-8152-4f9e-adf1-530299244421 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:42.359700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-80d28620-8152-4f9e-adf1-530299244421 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:42.359700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:42.359700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:42.359700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:42.359700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:42.359700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:42.359700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:42.359700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:42.359700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:42.359700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:42.359700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:42.359700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:42.359700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:42.359700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:42.359700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:42.361071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:42.361071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:42.361071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:42.361071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:42.361071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:42.361071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:42.361071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:42.361071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:42.361071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:42.361071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:42.361071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:42.361071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:42.361071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:42.361071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:42.361071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:42.361071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:42.361071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:42.362557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:42.362557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:42.362557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:42.362557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:42.362557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 346/664] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:42.362557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d28620-8152-4f9e-adf1-530299244421 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:42.366463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:42.477890 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421ddd9-b008-4c61-ab35-0e7a540bf872 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:42.502348 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d28620-8152-4f9e-adf1-530299244421 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:42.528848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ab9b5-bab8-4cc1-bbdd-00c29c574ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:42.529752 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ab9b5-bab8-4cc1-bbdd-00c29c574ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:42.530077 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ab9b5-bab8-4cc1-bbdd-00c29c574ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:42.530825 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ab9b5-bab8-4cc1-bbdd-00c29c574ec7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:42.544362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ab9b5-bab8-4cc1-bbdd-00c29c574ec7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:42.551211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96368f9-73ce-4533-9ba7-f3ee95b0e16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:42.551861 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96368f9-73ce-4533-9ba7-f3ee95b0e16f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:42.552131 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96368f9-73ce-4533-9ba7-f3ee95b0e16f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:42.552419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96368f9-73ce-4533-9ba7-f3ee95b0e16f None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:42.553967 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96368f9-73ce-4533-9ba7-f3ee95b0e16f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:42.554502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96368f9-73ce-4533-9ba7-f3ee95b0e16f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:42.556119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96368f9-73ce-4533-9ba7-f3ee95b0e16f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:42.561145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-ec2ab9b5-bab8-4cc1-bbdd-00c29c574ec7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:42.561145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:42.561145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:42.561145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:42.561145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:42.561145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:42.561145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:42.561145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:42.561145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:42.561145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:42.561145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:42.561145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:42.561145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:42.561145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:42.561145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:42.561145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:42.569410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:42.569410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:42.569410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:42.569410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:42.569410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:42.569410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:42.569410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:42.569410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:42.569410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:42.569410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:42.569410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:42.569410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:42.569410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:42.569410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:42.569410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:42.569410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:42.570530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:42.570530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:42.570530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:42.570530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:42.570530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:42.570530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:42.570530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:42.570530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:42.570530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:42.570530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:42.570530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:42.570530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:42.570530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 319/665] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:25:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:42.570530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ab9b5-bab8-4cc1-bbdd-00c29c574ec7 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:42.584537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 347/666] 10.222.0.57 () {56 vars in 1139 bytes} [Mon Jul 18 08:25:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:42.585371 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96368f9-73ce-4533-9ba7-f3ee95b0e16f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:42.636282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:42.664113 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:42.713615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96368f9-73ce-4533-9ba7-f3ee95b0e16f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:42.714060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ab9b5-bab8-4cc1-bbdd-00c29c574ec7 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:42.758225 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e59a40-946c-40e1-98e1-9b3c74a3e9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:42.759033 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e59a40-946c-40e1-98e1-9b3c74a3e9ad None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:42.759305 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e59a40-946c-40e1-98e1-9b3c74a3e9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:42.759603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e59a40-946c-40e1-98e1-9b3c74a3e9ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:42.766036 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb8021-5127-4f48-8c86-b61f77566848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:42.766714 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb8021-5127-4f48-8c86-b61f77566848 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:42.766996 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb8021-5127-4f48-8c86-b61f77566848 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:42.767311 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb8021-5127-4f48-8c86-b61f77566848 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:42.768071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb8021-5127-4f48-8c86-b61f77566848 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:42.768227 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e59a40-946c-40e1-98e1-9b3c74a3e9ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:42.768394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb8021-5127-4f48-8c86-b61f77566848 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:42.769604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e59a40-946c-40e1-98e1-9b3c74a3e9ad None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:42.769780 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb8021-5127-4f48-8c86-b61f77566848 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:42.786336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-93e59a40-946c-40e1-98e1-9b3c74a3e9ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:42.786336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:42.786336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:42.786336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:42.786336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:42.786336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:42.786336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:42.786336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:42.786336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:42.786336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:42.786336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:42.786336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:42.786336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:25:42.786336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:25:42.786336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:25:42.789805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:25:42.789805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:42.789805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:42.789805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:42.789805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:42.789805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:42.789805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:42.789805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:42.789805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:42.789805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:42.789805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:42.789805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:42.789805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:42.789805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:42.789805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:42.789805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:42.789805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:42.792152 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:25:42.792152 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:25:42.792152 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:25:42.792152 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:25:42.792152 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:25:42.792152 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:25:42.792152 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:42.792152 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:42.793711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 348/667] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:42.793711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb8021-5127-4f48-8c86-b61f77566848 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:42.794137 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 320/668] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:42.794137 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e59a40-946c-40e1-98e1-9b3c74a3e9ad None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:42.937056 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb8021-5127-4f48-8c86-b61f77566848 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:42.943060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e59a40-946c-40e1-98e1-9b3c74a3e9ad None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:42.946757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:42.966909 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:42.986988 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93a886-3d58-453e-a949-e3b751c60589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:42.988223 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93a886-3d58-453e-a949-e3b751c60589 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:42.988723 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93a886-3d58-453e-a949-e3b751c60589 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:42.989457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93a886-3d58-453e-a949-e3b751c60589 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:42.990559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93a886-3d58-453e-a949-e3b751c60589 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:42.990961 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93a886-3d58-453e-a949-e3b751c60589 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:42.992855 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93a886-3d58-453e-a949-e3b751c60589 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:42.996336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47184d48-0675-4577-b1b0-33a64e0ad397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:42.997385 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47184d48-0675-4577-b1b0-33a64e0ad397 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:42.997881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47184d48-0675-4577-b1b0-33a64e0ad397 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:42.998382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47184d48-0675-4577-b1b0-33a64e0ad397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:43.003817 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 349/669] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:25:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:43.004603 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93a886-3d58-453e-a949-e3b751c60589 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:43.108373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47184d48-0675-4577-b1b0-33a64e0ad397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:43.110689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47184d48-0675-4577-b1b0-33a64e0ad397 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:43.150379 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93a886-3d58-453e-a949-e3b751c60589 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:43.198941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a67418-3fc3-4833-af6f-0a56756efe18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:43.200147 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a67418-3fc3-4833-af6f-0a56756efe18 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:43.200435 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a67418-3fc3-4833-af6f-0a56756efe18 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:43.200797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a67418-3fc3-4833-af6f-0a56756efe18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:43.209942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a67418-3fc3-4833-af6f-0a56756efe18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:43.211450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a67418-3fc3-4833-af6f-0a56756efe18 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:43.227437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-a9a67418-3fc3-4833-af6f-0a56756efe18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:43.227437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:43.227437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:43.227437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:43.227437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:43.227437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:43.227437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:43.227437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:43.227437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:43.227437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:43.227437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:43.227437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:43.227437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:25:43.227437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:25:43.227437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:25:43.229230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:25:43.229230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:43.229230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:43.229230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:43.229230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:43.229230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:43.229230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:43.229230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:43.229230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:43.229230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:43.229230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:43.229230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:43.229230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:43.229230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:43.229230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:43.229230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:43.229230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:43.231200 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:25:43.231200 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:25:43.231200 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:25:43.231200 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:25:43.231200 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:25:43.231200 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:25:43.231200 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:43.231200 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:43.233741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 350/670] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:43.234354 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a67418-3fc3-4833-af6f-0a56756efe18 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:43.255966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 321/671] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:25:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:25:43.262153 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47184d48-0675-4577-b1b0-33a64e0ad397 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:43.265905 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:43.297840 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:43.366866 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a67418-3fc3-4833-af6f-0a56756efe18 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:43.408193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47184d48-0675-4577-b1b0-33a64e0ad397 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:43.411546 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461d8d1-7a8f-4c41-a6a9-d52b2b5498bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:43.414710 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461d8d1-7a8f-4c41-a6a9-d52b2b5498bb None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:43.415208 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461d8d1-7a8f-4c41-a6a9-d52b2b5498bb None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:43.415633 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461d8d1-7a8f-4c41-a6a9-d52b2b5498bb None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:43.416628 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461d8d1-7a8f-4c41-a6a9-d52b2b5498bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:43.417196 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461d8d1-7a8f-4c41-a6a9-d52b2b5498bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:43.419407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461d8d1-7a8f-4c41-a6a9-d52b2b5498bb None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:43.439654 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 351/672] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:25:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:43.441175 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461d8d1-7a8f-4c41-a6a9-d52b2b5498bb None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:43.451712 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd06b65-7429-45b8-91c9-9332b4963fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:43.452629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd06b65-7429-45b8-91c9-9332b4963fba None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:43.453011 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd06b65-7429-45b8-91c9-9332b4963fba None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:43.453384 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd06b65-7429-45b8-91c9-9332b4963fba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:43.466808 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd06b65-7429-45b8-91c9-9332b4963fba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:43.468492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd06b65-7429-45b8-91c9-9332b4963fba None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:43.487467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-afd06b65-7429-45b8-91c9-9332b4963fba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:43.487467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:43.487467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:43.487467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:43.487467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:43.487467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:43.487467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:43.487467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:43.487467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:43.487467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:43.487467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:43.487467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:43.487467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:43.487467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:43.487467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:43.489804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:43.489804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:43.489804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:43.489804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:43.489804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:43.489804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:43.489804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:43.489804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:43.489804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:43.489804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:43.489804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:43.489804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:43.489804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:43.489804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:43.489804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:43.489804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:43.489804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:43.500066 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:43.500066 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:43.500066 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:43.500066 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:43.500066 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 322/673] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:43.500066 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd06b65-7429-45b8-91c9-9332b4963fba None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:43.573619 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:43.588802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461d8d1-7a8f-4c41-a6a9-d52b2b5498bb None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:43.590570 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:43.641158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd06b65-7429-45b8-91c9-9332b4963fba None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:43.645286 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133da2a-8009-46e3-aa13-d7d494137692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:43.646488 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133da2a-8009-46e3-aa13-d7d494137692 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:43.647636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133da2a-8009-46e3-aa13-d7d494137692 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:43.648103 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133da2a-8009-46e3-aa13-d7d494137692 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:43.649625 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0133da2a-8009-46e3-aa13-d7d494137692 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:43.650168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0133da2a-8009-46e3-aa13-d7d494137692 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:43.652968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0133da2a-8009-46e3-aa13-d7d494137692 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:43.675442 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 352/674] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:25:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:43.676810 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133da2a-8009-46e3-aa13-d7d494137692 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:43.682950 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f0c47-0476-4c33-8d62-457e978725c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:43.683807 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f0c47-0476-4c33-8d62-457e978725c3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:43.684108 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f0c47-0476-4c33-8d62-457e978725c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:43.684366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f0c47-0476-4c33-8d62-457e978725c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:43.699191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f0c47-0476-4c33-8d62-457e978725c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:43.702183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f0c47-0476-4c33-8d62-457e978725c3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:43.726380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-057f0c47-0476-4c33-8d62-457e978725c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:43.726380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:43.726380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:43.726380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:43.726380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:43.726380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:43.726380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:43.726380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:43.726380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:43.726380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:43.726380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:43.726380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:43.726380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:43.726380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:43.726380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:43.728231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:43.728231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:43.728231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:43.728231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:43.728231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:43.728231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:43.728231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:43.728231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:43.728231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:43.728231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:43.728231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:43.728231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:43.728231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:43.728231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:43.728231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:43.728231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:43.728231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:43.729855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:43.729855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:43.729855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:43.729855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:43.729855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 323/675] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:43.729855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f0c47-0476-4c33-8d62-457e978725c3 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:43.822000 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133da2a-8009-46e3-aa13-d7d494137692 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:43.874285 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f0c47-0476-4c33-8d62-457e978725c3 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:43.876918 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d7ac5-8280-449b-8b34-3f91036ad854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:43.877791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d7ac5-8280-449b-8b34-3f91036ad854 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:43.878262 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d7ac5-8280-449b-8b34-3f91036ad854 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:43.878637 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d7ac5-8280-449b-8b34-3f91036ad854 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:43.880099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d7ac5-8280-449b-8b34-3f91036ad854 None admin] RBAC: Adding query filter params (role.id=9fc4c3dcb90945ebb4eb5566e62e7833, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=d5f9a129021445ae8154a5e6941291e0) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:43.880615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d7ac5-8280-449b-8b34-3f91036ad854 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:43.883769 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d7ac5-8280-449b-8b34-3f91036ad854 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:43.890626 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:43.894064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 353/676] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:25:43 2022] GET /identity/v3/role_assignments?user.id=d5f9a129021445ae8154a5e6941291e0&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=9fc4c3dcb90945ebb4eb5566e62e7833 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:43.896114 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571b5f3-e66f-458d-9c64-0914c651acb7 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:43.896619 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571b5f3-e66f-458d-9c64-0914c651acb7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:43.897586 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571b5f3-e66f-458d-9c64-0914c651acb7 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:43.900276 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 354/677] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:43.901292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571b5f3-e66f-458d-9c64-0914c651acb7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:43.939331 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a560e3c-efc4-4259-9954-332fb40c48b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:43.940298 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a560e3c-efc4-4259-9954-332fb40c48b2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:43.940719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a560e3c-efc4-4259-9954-332fb40c48b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:43.941140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a560e3c-efc4-4259-9954-332fb40c48b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:43.942103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a560e3c-efc4-4259-9954-332fb40c48b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:43.942621 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a560e3c-efc4-4259-9954-332fb40c48b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:43.945227 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a560e3c-efc4-4259-9954-332fb40c48b2 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:43.953802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:43.962770 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 324/678] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:43.964278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a560e3c-efc4-4259-9954-332fb40c48b2 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:44.102385 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a560e3c-efc4-4259-9954-332fb40c48b2 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:44.112330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571b5f3-e66f-458d-9c64-0914c651acb7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:44.160747 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b5919-d6bb-4225-b53c-ab11f61044c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:44.161752 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b5919-d6bb-4225-b53c-ab11f61044c1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:44.162458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b5919-d6bb-4225-b53c-ab11f61044c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:44.162806 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b5919-d6bb-4225-b53c-ab11f61044c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:44.164006 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b5919-d6bb-4225-b53c-ab11f61044c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:44.164507 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b5919-d6bb-4225-b53c-ab11f61044c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:44.166610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b5919-d6bb-4225-b53c-ab11f61044c1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:44.179604 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef6b7c-ca56-4cf9-8bcf-efc94f133b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:44.182330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef6b7c-ca56-4cf9-8bcf-efc94f133b86 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:44.183088 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef6b7c-ca56-4cf9-8bcf-efc94f133b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:44.183419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef6b7c-ca56-4cf9-8bcf-efc94f133b86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:44.185255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef6b7c-ca56-4cf9-8bcf-efc94f133b86 None admin] RBAC: Adding query filter params (role.id=9fc4c3dcb90945ebb4eb5566e62e7833, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=694088ed30b94bad9d3b16f3766252df) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:44.185554 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef6b7c-ca56-4cf9-8bcf-efc94f133b86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:44.186616 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 325/679] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:44.188132 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef6b7c-ca56-4cf9-8bcf-efc94f133b86 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:44.188500 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282bd60-034f-480e-872a-b53a0b5a54c3 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:44.188955 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282bd60-034f-480e-872a-b53a0b5a54c3 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:44.189396 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282bd60-034f-480e-872a-b53a0b5a54c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:44.200461 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 355/680] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:25:43 2022] GET /identity/v3/role_assignments?user.id=694088ed30b94bad9d3b16f3766252df&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=9fc4c3dcb90945ebb4eb5566e62e7833 => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:44.200461 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef6b7c-ca56-4cf9-8bcf-efc94f133b86 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:44.219920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0282bd60-034f-480e-872a-b53a0b5a54c3 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:44.222733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0282bd60-034f-480e-872a-b53a0b5a54c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:44.284915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:44.385095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 326/681] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:44.386576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282bd60-034f-480e-872a-b53a0b5a54c3 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:44.408435 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef6b7c-ca56-4cf9-8bcf-efc94f133b86 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:44.425769 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:44.488218 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525eb0c-108c-4dd7-9fd3-5fa655128268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:44.490069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525eb0c-108c-4dd7-9fd3-5fa655128268 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:44.491643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525eb0c-108c-4dd7-9fd3-5fa655128268 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:44.492263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525eb0c-108c-4dd7-9fd3-5fa655128268 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:44.506682 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f525eb0c-108c-4dd7-9fd3-5fa655128268 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:44.509502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f525eb0c-108c-4dd7-9fd3-5fa655128268 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:44.536746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f525eb0c-108c-4dd7-9fd3-5fa655128268 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 08:25:44.536746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:44.536746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:44.536746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:44.536746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:44.536746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:44.536746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:44.536746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:44.536746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:44.536746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:44.536746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:44.536746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:44.536746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:25:44.536746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:25:44.536746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:25:44.536746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:25:44.540095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:44.540095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:44.540095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:25:44.540095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:44.540095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:25:44.540095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:44.540095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:44.540095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:44.540095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:44.540095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:44.540095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:44.540095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:44.540095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:44.540095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:44.540095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:44.540095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:44.540095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:44.540095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:44.543205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:44.543205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:44.543205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:25:44.543205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:25:44.543205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:25:44.543205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:25:44.543205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:25:44.543205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:25:44.543205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 08:25:44.543205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:44.543205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 356/682] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:25:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:44.545550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525eb0c-108c-4dd7-9fd3-5fa655128268 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:44.604887 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282bd60-034f-480e-872a-b53a0b5a54c3 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:44.667217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1e269-0db2-4968-8e40-0d5ace2c40f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:44.668672 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1e269-0db2-4968-8e40-0d5ace2c40f5 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:44.669156 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1e269-0db2-4968-8e40-0d5ace2c40f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:44.669648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1e269-0db2-4968-8e40-0d5ace2c40f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:44.683869 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c1e269-0db2-4968-8e40-0d5ace2c40f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:44.686658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c1e269-0db2-4968-8e40-0d5ace2c40f5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:44.717089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-60c1e269-0db2-4968-8e40-0d5ace2c40f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:44.717089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:44.717089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:44.717089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:44.717089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:44.717089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:44.717089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:44.717089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:44.717089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:44.717089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:44.717089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:44.717089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:44.717089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:25:44.717089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:25:44.717089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:25:44.718772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:25:44.718772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:44.718772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:44.718772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:44.718772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:44.718772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:44.718772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:44.718772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:44.718772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:44.718772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:44.718772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:44.718772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:44.718772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:44.718772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:44.718772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:44.718772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:44.718772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:44.721114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:25:44.721114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:25:44.721114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:25:44.721114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:25:44.721114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:25:44.721114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:25:44.721114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:44.721114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:44.721114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 327/683] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:44.721114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1e269-0db2-4968-8e40-0d5ace2c40f5 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:44.766550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525eb0c-108c-4dd7-9fd3-5fa655128268 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:44.772741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:44.846770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd07d49-5af7-427b-8eb9-e429668a7de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:44.846770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd07d49-5af7-427b-8eb9-e429668a7de1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:44.846770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd07d49-5af7-427b-8eb9-e429668a7de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:44.846770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd07d49-5af7-427b-8eb9-e429668a7de1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:44.871497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd07d49-5af7-427b-8eb9-e429668a7de1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:44.888831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-6fd07d49-5af7-427b-8eb9-e429668a7de1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:25:44.888831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:44.888831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:44.888831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:44.888831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:44.888831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:44.888831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:44.888831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:44.888831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:44.888831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:44.888831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:44.888831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:44.888831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:44.888831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:44.888831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:44.888831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:44.890711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:44.890711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:44.890711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:44.890711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:44.890711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:44.890711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:44.890711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:44.890711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:44.890711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:44.890711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:44.890711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:44.890711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:44.890711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:44.890711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:44.890711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:44.890711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:44.893295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:44.893295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:44.893295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:44.893295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:44.893295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:44.893295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:44.893295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:44.893295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:44.893295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:44.893295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:44.893295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:25:44.893295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:44.893295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 357/684] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:25:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:44.893295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd07d49-5af7-427b-8eb9-e429668a7de1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:44.918784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:44.952882 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1e269-0db2-4968-8e40-0d5ace2c40f5 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:45.025576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4a090-1288-46f6-8b00-5916f0821f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:45.027342 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4a090-1288-46f6-8b00-5916f0821f7a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:45.028220 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4a090-1288-46f6-8b00-5916f0821f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:45.029152 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4a090-1288-46f6-8b00-5916f0821f7a None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:45.031307 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4a090-1288-46f6-8b00-5916f0821f7a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:45.032601 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4a090-1288-46f6-8b00-5916f0821f7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:45.036160 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4a090-1288-46f6-8b00-5916f0821f7a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:45.051087 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd07d49-5af7-427b-8eb9-e429668a7de1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:45.076770 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 328/685] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Jul 18 08:25:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:45.078887 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4a090-1288-46f6-8b00-5916f0821f7a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:45.099754 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae773d3-8568-4a04-9d55-dfcafdaea475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:45.100949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae773d3-8568-4a04-9d55-dfcafdaea475 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:45.101233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae773d3-8568-4a04-9d55-dfcafdaea475 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:45.101555 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae773d3-8568-4a04-9d55-dfcafdaea475 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:45.102491 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae773d3-8568-4a04-9d55-dfcafdaea475 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:45.102810 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae773d3-8568-4a04-9d55-dfcafdaea475 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:45.104378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae773d3-8568-4a04-9d55-dfcafdaea475 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:45.132040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 358/686] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:25:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:45.135406 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae773d3-8568-4a04-9d55-dfcafdaea475 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:45.213088 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:45.242245 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4a090-1288-46f6-8b00-5916f0821f7a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:45.284578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:45.305675 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97421301-6fdd-49a8-934b-55f2813f9f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:45.306546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97421301-6fdd-49a8-934b-55f2813f9f72 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:45.307019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97421301-6fdd-49a8-934b-55f2813f9f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:45.307897 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97421301-6fdd-49a8-934b-55f2813f9f72 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:45.308313 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97421301-6fdd-49a8-934b-55f2813f9f72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:45.308750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97421301-6fdd-49a8-934b-55f2813f9f72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:45.310402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97421301-6fdd-49a8-934b-55f2813f9f72 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:45.321981 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 329/687] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:25:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:45.323096 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97421301-6fdd-49a8-934b-55f2813f9f72 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:45.350620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae773d3-8568-4a04-9d55-dfcafdaea475 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:45.437857 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82028dd0-ec2e-42f1-b8d8-7b4e77939e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:45.440225 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82028dd0-ec2e-42f1-b8d8-7b4e77939e19 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:45.441595 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82028dd0-ec2e-42f1-b8d8-7b4e77939e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:45.442294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82028dd0-ec2e-42f1-b8d8-7b4e77939e19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:45.478178 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97421301-6fdd-49a8-934b-55f2813f9f72 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:45.529088 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666e30a-0e56-4829-9721-541a6b5a5e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:45.534464 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666e30a-0e56-4829-9721-541a6b5a5e49 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:45.534464 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666e30a-0e56-4829-9721-541a6b5a5e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:45.534464 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666e30a-0e56-4829-9721-541a6b5a5e49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:45.534972 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666e30a-0e56-4829-9721-541a6b5a5e49 None admin] RBAC: Adding query filter params (role.id=9fc4c3dcb90945ebb4eb5566e62e7833, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=24a6b794f5774e35b67e9ec4358ffc66) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:45.535549 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666e30a-0e56-4829-9721-541a6b5a5e49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:45.540519 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666e30a-0e56-4829-9721-541a6b5a5e49 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:45.558238 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 330/688] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:25:45 2022] GET /identity/v3/role_assignments?user.id=24a6b794f5774e35b67e9ec4358ffc66&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=9fc4c3dcb90945ebb4eb5566e62e7833 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:45.559869 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666e30a-0e56-4829-9721-541a6b5a5e49 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:45.565213 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:45.585570 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82028dd0-ec2e-42f1-b8d8-7b4e77939e19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:45.587388 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82028dd0-ec2e-42f1-b8d8-7b4e77939e19 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:45.801432 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:45.814071 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666e30a-0e56-4829-9721-541a6b5a5e49 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:45.881746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 359/689] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:25:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 749 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:25:45.888455 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82028dd0-ec2e-42f1-b8d8-7b4e77939e19 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:45.918036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3bfce-649d-4bc4-bdd2-0cd71545ba16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:45.919332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3bfce-649d-4bc4-bdd2-0cd71545ba16 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:45.919987 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3bfce-649d-4bc4-bdd2-0cd71545ba16 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:45.920460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3bfce-649d-4bc4-bdd2-0cd71545ba16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:45.937273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a3bfce-649d-4bc4-bdd2-0cd71545ba16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:45.938545 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a3bfce-649d-4bc4-bdd2-0cd71545ba16 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:45.960892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-56a3bfce-649d-4bc4-bdd2-0cd71545ba16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:45.960892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:45.960892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:45.960892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:45.960892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:45.960892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:45.960892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:45.960892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:45.960892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:45.960892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:45.960892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:45.960892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:45.960892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:45.960892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:45.960892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:45.962998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:45.962998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:45.962998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:45.962998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:45.962998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:45.962998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:45.962998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:45.962998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:45.962998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:45.962998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:45.962998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:45.962998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:45.962998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:45.962998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:45.962998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:45.962998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:45.962998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:45.964823 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:45.964823 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:45.964823 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:45.964823 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:45.964823 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 331/690] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:45.969968 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3bfce-649d-4bc4-bdd2-0cd71545ba16 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:46.053585 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82028dd0-ec2e-42f1-b8d8-7b4e77939e19 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:46.085340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:46.099170 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09c151-620c-4ba8-b7ce-44f838400e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:46.101202 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09c151-620c-4ba8-b7ce-44f838400e3a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:46.107615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09c151-620c-4ba8-b7ce-44f838400e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:46.107615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09c151-620c-4ba8-b7ce-44f838400e3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:46.119953 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09c151-620c-4ba8-b7ce-44f838400e3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:46.122454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09c151-620c-4ba8-b7ce-44f838400e3a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:46.158281 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3bfce-649d-4bc4-bdd2-0cd71545ba16 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:46.165293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-ed09c151-620c-4ba8-b7ce-44f838400e3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:46.165293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:46.165293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:46.165293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:46.165293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:46.165293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:46.165293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:46.165293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:46.165293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:46.165293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:46.165293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:46.165293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:46.165293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:46.165293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:46.165293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:46.166250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:46.166250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:46.166250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:46.166250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:46.166250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:46.166250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:46.166250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:46.166250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:46.166250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:46.166250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:46.166250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:46.166250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:46.166250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:46.166250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:46.166250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:46.166250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:46.166250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:46.167224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:46.167224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:46.167224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:46.167224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:46.176032 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 360/691] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:46.178594 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09c151-620c-4ba8-b7ce-44f838400e3a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:46.241246 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:46.269644 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc2be4-4fb3-4e08-9e25-6b75b1fd3c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:46.270575 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc2be4-4fb3-4e08-9e25-6b75b1fd3c6a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:46.270865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc2be4-4fb3-4e08-9e25-6b75b1fd3c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:46.271166 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc2be4-4fb3-4e08-9e25-6b75b1fd3c6a None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:46.273160 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc2be4-4fb3-4e08-9e25-6b75b1fd3c6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:46.273747 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc2be4-4fb3-4e08-9e25-6b75b1fd3c6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:46.276430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc2be4-4fb3-4e08-9e25-6b75b1fd3c6a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:46.315845 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 332/692] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:46.324130 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc2be4-4fb3-4e08-9e25-6b75b1fd3c6a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:46.390087 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09c151-620c-4ba8-b7ce-44f838400e3a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:46.455160 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ed2ad-85bd-4330-9bd2-e58ab2c9fd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:46.456242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ed2ad-85bd-4330-9bd2-e58ab2c9fd0e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:46.456731 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ed2ad-85bd-4330-9bd2-e58ab2c9fd0e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:46.457298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ed2ad-85bd-4330-9bd2-e58ab2c9fd0e None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:46.458286 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ed2ad-85bd-4330-9bd2-e58ab2c9fd0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:46.458821 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ed2ad-85bd-4330-9bd2-e58ab2c9fd0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:46.461253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ed2ad-85bd-4330-9bd2-e58ab2c9fd0e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:46.480629 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 361/693] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:46.493378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ed2ad-85bd-4330-9bd2-e58ab2c9fd0e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:46.516307 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc2be4-4fb3-4e08-9e25-6b75b1fd3c6a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:46.542323 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:46.578853 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910265d4-6dbf-4c90-a084-85c8468601ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:46.579602 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910265d4-6dbf-4c90-a084-85c8468601ed None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:46.579915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910265d4-6dbf-4c90-a084-85c8468601ed None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:46.580193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910265d4-6dbf-4c90-a084-85c8468601ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:46.598290 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910265d4-6dbf-4c90-a084-85c8468601ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:46.600375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910265d4-6dbf-4c90-a084-85c8468601ed None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:46.633103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-910265d4-6dbf-4c90-a084-85c8468601ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 08:25:46.633103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:46.633103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:46.633103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:46.633103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:46.633103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:46.633103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:46.633103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:46.633103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:46.633103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:46.633103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:46.633103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:46.633103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:25:46.633103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:25:46.633103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:25:46.633103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:25:46.634113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:46.634113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:46.634113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:25:46.634113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:46.634113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:25:46.634113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:46.634113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:46.634113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:46.634113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:46.634113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:46.634113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:46.634113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:46.634113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:46.634113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:46.634113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:46.634113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:46.634113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:46.634113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:46.635044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:46.635044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:46.635044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:25:46.635044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:25:46.635044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:25:46.635044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:25:46.635044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:25:46.635044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:25:46.635044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 08:25:46.635044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:46.647007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 333/694] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 18 08:25:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:25:46.656023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910265d4-6dbf-4c90-a084-85c8468601ed None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:46.657814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:46.680502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ed2ad-85bd-4330-9bd2-e58ab2c9fd0e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:46.731455 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f672d5a-8427-4c57-89b4-f6f3d2511f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:46.732376 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f672d5a-8427-4c57-89b4-f6f3d2511f54 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:46.732662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f672d5a-8427-4c57-89b4-f6f3d2511f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:46.732940 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f672d5a-8427-4c57-89b4-f6f3d2511f54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:46.749137 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f672d5a-8427-4c57-89b4-f6f3d2511f54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:46.751138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f672d5a-8427-4c57-89b4-f6f3d2511f54 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:46.779002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-9f672d5a-8427-4c57-89b4-f6f3d2511f54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:46.779002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:46.779002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:46.779002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:46.779002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:46.779002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:46.779002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:46.779002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:46.779002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:46.779002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:46.779002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:46.779002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:46.779002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:25:46.779002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:25:46.779002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:25:46.780013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:25:46.780013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:46.780013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:46.780013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:46.780013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:46.780013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:46.780013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:46.780013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:46.780013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:46.780013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:46.780013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:46.780013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:46.780013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:46.780013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:46.780013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:46.780013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:46.780013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:46.780891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:25:46.780891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:25:46.780891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:25:46.780891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:25:46.780891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:25:46.780891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:25:46.780891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:46.780891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:46.780891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 362/695] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:46.786728 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f672d5a-8427-4c57-89b4-f6f3d2511f54 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:46.907142 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f672d5a-8427-4c57-89b4-f6f3d2511f54 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:46.958413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677243cd-0acc-46bc-9a88-3936a54e65bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:46.959143 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677243cd-0acc-46bc-9a88-3936a54e65bb None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:46.959431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677243cd-0acc-46bc-9a88-3936a54e65bb None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:46.959729 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677243cd-0acc-46bc-9a88-3936a54e65bb None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:46.961210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677243cd-0acc-46bc-9a88-3936a54e65bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:46.961533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677243cd-0acc-46bc-9a88-3936a54e65bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:46.963328 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677243cd-0acc-46bc-9a88-3936a54e65bb None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:46.988470 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 363/696] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:25:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:46.995739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677243cd-0acc-46bc-9a88-3936a54e65bb None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:47.002409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:47.019904 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910265d4-6dbf-4c90-a084-85c8468601ed None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:47.111497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb59af-7e9c-4500-a8bc-9d0bb7936bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:47.112687 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb59af-7e9c-4500-a8bc-9d0bb7936bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:47.113193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb59af-7e9c-4500-a8bc-9d0bb7936bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:47.113688 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb59af-7e9c-4500-a8bc-9d0bb7936bf8 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:47.115133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb59af-7e9c-4500-a8bc-9d0bb7936bf8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:47.116027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb59af-7e9c-4500-a8bc-9d0bb7936bf8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:47.124149 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb59af-7e9c-4500-a8bc-9d0bb7936bf8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:47.141647 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677243cd-0acc-46bc-9a88-3936a54e65bb None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:47.182089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 334/697] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 18 08:25:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 527 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:47.183474 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e8c2b-438b-43ca-85fa-cb129de914ae None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:47.183794 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e8c2b-438b-43ca-85fa-cb129de914ae None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:47.184077 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e8c2b-438b-43ca-85fa-cb129de914ae None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:47.185621 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 335/698] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:47.192059 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71dc91-be3e-4825-b71a-db4609eb1e3b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:47.192227 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99215860-82b1-4450-8665-21db04728f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:47.192347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71dc91-be3e-4825-b71a-db4609eb1e3b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:47.192618 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71dc91-be3e-4825-b71a-db4609eb1e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:47.194107 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99215860-82b1-4450-8665-21db04728f60 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:47.194596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99215860-82b1-4450-8665-21db04728f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:47.195117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99215860-82b1-4450-8665-21db04728f60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:47.196222 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99215860-82b1-4450-8665-21db04728f60 None admin] RBAC: Adding query filter params (role.id=9fc4c3dcb90945ebb4eb5566e62e7833, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=09a5c2e804504038817cad76544dce5f) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:47.196749 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99215860-82b1-4450-8665-21db04728f60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:47.198572 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99215860-82b1-4450-8665-21db04728f60 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:47.205981 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 364/699] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:25:46 2022] GET /identity/v3/role_assignments?user.id=09a5c2e804504038817cad76544dce5f&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=9fc4c3dcb90945ebb4eb5566e62e7833 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:47.207168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99215860-82b1-4450-8665-21db04728f60 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:47.229052 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-ac71dc91-be3e-4825-b71a-db4609eb1e3b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:47.232615 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ac71dc91-be3e-4825-b71a-db4609eb1e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:47.298780 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:47.379867 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:47.440438 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 336/700] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:25:47.475198 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99215860-82b1-4450-8665-21db04728f60 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:47.495232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71dc91-be3e-4825-b71a-db4609eb1e3b None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:47.558137 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66158dd9-5268-4401-8693-3ba116cc77f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:47.560045 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66158dd9-5268-4401-8693-3ba116cc77f0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:47.560869 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66158dd9-5268-4401-8693-3ba116cc77f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:47.561692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66158dd9-5268-4401-8693-3ba116cc77f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:47.620921 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71dc91-be3e-4825-b71a-db4609eb1e3b None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:47.684055 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f7020-e271-422b-926a-b7d30a0b200b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:47.686280 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f7020-e271-422b-926a-b7d30a0b200b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:47.686776 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f7020-e271-422b-926a-b7d30a0b200b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:47.687272 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f7020-e271-422b-926a-b7d30a0b200b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:47.688868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f7020-e271-422b-926a-b7d30a0b200b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:47.689772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66158dd9-5268-4401-8693-3ba116cc77f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:47.691218 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f7020-e271-422b-926a-b7d30a0b200b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:47.692694 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66158dd9-5268-4401-8693-3ba116cc77f0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:47.730911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-c03f7020-e271-422b-926a-b7d30a0b200b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 08:25:47.730911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:47.730911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:25:47.730911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:25:47.730911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:25:47.730911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:25:47.730911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:25:47.730911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:25:47.730911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 08:25:47.730911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:47.730911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:25:47.730911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:47.730911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:47.730911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:47.730911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:47.730911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:47.732100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:47.732100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:47.732100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:47.732100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:47.732100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:47.732100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:47.732100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:47.732100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:25:47.732100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:25:47.732100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:25:47.732100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:25:47.732100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:47.732100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:47.732100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:47.732100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:47.732100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:47.733042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:47.733042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:47.733042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:47.733042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:47.733042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:47.733042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:47.733042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:47.733042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:47.733042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:47.733042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:47.733042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:47.733042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:25:47.733042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:25:47.733042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 08:25:47.733042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:47.734471 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 337/701] 10.222.0.57 () {56 vars in 1129 bytes} [Mon Jul 18 08:25:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:25:47.736298 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00064300-be07-432d-b072-eef156c86752 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:47.736897 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00064300-be07-432d-b072-eef156c86752 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:47.738358 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00064300-be07-432d-b072-eef156c86752 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:47.742659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 338/702] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:47.751055 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00064300-be07-432d-b072-eef156c86752 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:47.808451 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:47.827463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:47.908064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 365/703] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:25:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:25:47.910921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749094e-f5bd-4030-91f7-e9a086a08933 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:47.913923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749094e-f5bd-4030-91f7-e9a086a08933 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:47.914820 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749094e-f5bd-4030-91f7-e9a086a08933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:47.946491 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8749094e-f5bd-4030-91f7-e9a086a08933 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:47.949006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8749094e-f5bd-4030-91f7-e9a086a08933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:47.977050 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00064300-be07-432d-b072-eef156c86752 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:48.027708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6dbd7-d760-4c69-81e1-ab8e9b434ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:48.028790 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6dbd7-d760-4c69-81e1-ab8e9b434ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:48.028790 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6dbd7-d760-4c69-81e1-ab8e9b434ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:48.029121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6dbd7-d760-4c69-81e1-ab8e9b434ce3 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:48.029797 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6dbd7-d760-4c69-81e1-ab8e9b434ce3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:48.030106 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6dbd7-d760-4c69-81e1-ab8e9b434ce3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:48.031548 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6dbd7-d760-4c69-81e1-ab8e9b434ce3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:48.038508 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 339/704] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Jul 18 08:25:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:48.039454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6dbd7-d760-4c69-81e1-ab8e9b434ce3 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:48.095658 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 366/705] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:48.097457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8749094e-f5bd-4030-91f7-e9a086a08933 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:48.198725 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:48.213816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6dbd7-d760-4c69-81e1-ab8e9b434ce3 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:48.267646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f963984-a5af-48cf-bddf-2b72672bbb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:48.268630 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f963984-a5af-48cf-bddf-2b72672bbb2e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:48.268823 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f963984-a5af-48cf-bddf-2b72672bbb2e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:48.269119 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f963984-a5af-48cf-bddf-2b72672bbb2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:48.282312 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f963984-a5af-48cf-bddf-2b72672bbb2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:48.283870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f963984-a5af-48cf-bddf-2b72672bbb2e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:48.306122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-4f963984-a5af-48cf-bddf-2b72672bbb2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:48.306122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:48.306122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:48.306122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:48.306122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:48.306122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:48.306122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:48.306122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:48.306122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:48.306122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:48.306122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:48.306122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:48.306122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:48.306122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:48.306122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:48.308086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:48.308086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:48.308086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:48.308086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:48.308086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:48.308086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:48.308086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:48.308086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:48.308086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:48.308086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:48.308086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:48.308086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:48.308086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:48.308086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:48.308086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:48.308086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:48.308086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:48.310195 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:48.310195 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:48.310195 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:48.310195 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:48.312242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 340/706] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:48.313220 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f963984-a5af-48cf-bddf-2b72672bbb2e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:48.342048 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:48.355448 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8749094e-f5bd-4030-91f7-e9a086a08933 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:48.435176 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b2685-d50b-4377-9b0a-3229091becc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:48.435943 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b2685-d50b-4377-9b0a-3229091becc3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:48.436241 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b2685-d50b-4377-9b0a-3229091becc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:48.436547 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b2685-d50b-4377-9b0a-3229091becc3 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:48.437280 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b2685-d50b-4377-9b0a-3229091becc3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:48.437588 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b2685-d50b-4377-9b0a-3229091becc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:48.438997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b2685-d50b-4377-9b0a-3229091becc3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:48.446500 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 367/707] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Jul 18 08:25:48 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:48.447443 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b2685-d50b-4377-9b0a-3229091becc3 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:48.492610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f963984-a5af-48cf-bddf-2b72672bbb2e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:48.538553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829f78f-5000-481d-b89a-b6741f548784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:48.539466 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829f78f-5000-481d-b89a-b6741f548784 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:48.539796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829f78f-5000-481d-b89a-b6741f548784 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:48.540038 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829f78f-5000-481d-b89a-b6741f548784 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:48.552666 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829f78f-5000-481d-b89a-b6741f548784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:48.555395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829f78f-5000-481d-b89a-b6741f548784 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:48.557541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:48.576283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-d829f78f-5000-481d-b89a-b6741f548784 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:48.576283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:48.576283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:48.576283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:48.576283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:48.576283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:48.576283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:48.576283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:48.576283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:48.576283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:48.576283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:48.576283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:48.576283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:48.576283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:48.576283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:48.582495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:48.582495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:48.582495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:48.582495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:48.582495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:48.582495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:48.582495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:48.582495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:48.582495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:48.582495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:48.582495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:48.582495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:48.582495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:48.582495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:48.582495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:48.582495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:48.582495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:48.583417 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:48.583417 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:48.583417 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:48.583417 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:48.583417 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 341/708] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:48.587506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829f78f-5000-481d-b89a-b6741f548784 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:48.670849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b2685-d50b-4377-9b0a-3229091becc3 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:48.707222 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829f78f-5000-481d-b89a-b6741f548784 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:48.721793 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18b93a-a990-4e7f-a78d-dcfd0bf7e648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:48.722493 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18b93a-a990-4e7f-a78d-dcfd0bf7e648 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:48.722779 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18b93a-a990-4e7f-a78d-dcfd0bf7e648 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:48.723058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18b93a-a990-4e7f-a78d-dcfd0bf7e648 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:48.723863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18b93a-a990-4e7f-a78d-dcfd0bf7e648 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:48.724177 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18b93a-a990-4e7f-a78d-dcfd0bf7e648 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:48.726961 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18b93a-a990-4e7f-a78d-dcfd0bf7e648 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:48.743174 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 368/709] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:48.744574 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8229ac-c03e-4db3-bf9e-c8858aaef920 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:48.744933 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8229ac-c03e-4db3-bf9e-c8858aaef920 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:48.745213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8229ac-c03e-4db3-bf9e-c8858aaef920 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:48.746793 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 369/710] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:48.752704 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb43fba-490c-448e-8274-c23e505b2cf9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:48.753055 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb43fba-490c-448e-8274-c23e505b2cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:48.753377 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb43fba-490c-448e-8274-c23e505b2cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:48.757184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7d130-2857-4fa9-a410-75e706c01840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:48.757879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7d130-2857-4fa9-a410-75e706c01840 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:48.758165 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7d130-2857-4fa9-a410-75e706c01840 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:48.758446 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7d130-2857-4fa9-a410-75e706c01840 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:48.759196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7d130-2857-4fa9-a410-75e706c01840 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:48.759502 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7d130-2857-4fa9-a410-75e706c01840 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:48.761070 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7d130-2857-4fa9-a410-75e706c01840 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:48.774237 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-dcb43fba-490c-448e-8274-c23e505b2cf9 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:48.775351 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 342/711] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:25:48.775547 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-dcb43fba-490c-448e-8274-c23e505b2cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:48.780563 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7d130-2857-4fa9-a410-75e706c01840 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:48.795395 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:48.848211 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:48.897701 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 370/712] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:25:48.899354 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb43fba-490c-448e-8274-c23e505b2cf9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:48.901689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7d130-2857-4fa9-a410-75e706c01840 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:48.940864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906645f-3840-4269-9c91-1ef25c86863a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:48.941806 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906645f-3840-4269-9c91-1ef25c86863a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:48.942155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906645f-3840-4269-9c91-1ef25c86863a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:48.942504 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906645f-3840-4269-9c91-1ef25c86863a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:48.950663 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0906645f-3840-4269-9c91-1ef25c86863a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:48.952401 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0906645f-3840-4269-9c91-1ef25c86863a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:48.968830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-0906645f-3840-4269-9c91-1ef25c86863a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:48.968830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:48.968830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:48.968830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:48.968830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:48.968830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:48.968830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:48.968830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:48.968830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:48.968830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:48.968830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:48.968830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:48.968830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:25:48.968830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:25:48.968830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:25:48.970719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:25:48.970719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:48.970719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:48.970719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:48.970719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:48.970719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:48.970719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:48.970719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:48.970719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:48.970719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:48.970719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:48.970719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:48.970719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:48.970719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:48.970719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:48.970719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:48.970719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:48.972903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:25:48.972903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:25:48.972903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:25:48.972903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:25:48.972903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:25:48.972903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:25:48.972903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:48.972903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:48.972903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 343/713] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:48.977649 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906645f-3840-4269-9c91-1ef25c86863a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:49.094896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906645f-3840-4269-9c91-1ef25c86863a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:49.104181 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb43fba-490c-448e-8274-c23e505b2cf9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:49.112437 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:49.137111 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391e8d4-3f8f-4751-bbc4-d14b47fc568d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:49.138691 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391e8d4-3f8f-4751-bbc4-d14b47fc568d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:49.139085 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391e8d4-3f8f-4751-bbc4-d14b47fc568d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:49.139459 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391e8d4-3f8f-4751-bbc4-d14b47fc568d None admin] PATH_INFO: `/v3/services/network` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:49.140602 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391e8d4-3f8f-4751-bbc4-d14b47fc568d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:49.142801 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391e8d4-3f8f-4751-bbc4-d14b47fc568d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:49.160194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-3391e8d4-3f8f-4751-bbc4-d14b47fc568d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 08:25:49.160194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:49.160194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:25:49.160194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:25:49.160194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:25:49.160194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:25:49.160194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:25:49.160194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:25:49.160194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 08:25:49.160194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:49.160194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:25:49.160194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:49.160194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:49.160194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:49.160194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:49.160194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:49.160194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:49.162246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:49.162246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:49.162246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:49.162246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:49.162246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:49.162246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:49.162246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:25:49.162246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:25:49.162246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:25:49.162246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:25:49.162246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:49.162246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:49.162246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:49.162246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:49.162246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:49.162246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:49.164209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:49.164209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:49.164209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:49.164209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:49.164209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:49.164209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:49.164209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:49.164209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:49.164209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:49.164209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:49.164209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:25:49.164209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:25:49.164209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 08:25:49.164209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:49.164209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 344/714] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:48 2022] GET /identity/v3/services/network => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:49.165876 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391e8d4-3f8f-4751-bbc4-d14b47fc568d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:49.189477 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf648b-3c72-4dad-ad67-dd5e3ed8b59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:49.190778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf648b-3c72-4dad-ad67-dd5e3ed8b59b None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:49.191366 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf648b-3c72-4dad-ad67-dd5e3ed8b59b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:49.192958 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf648b-3c72-4dad-ad67-dd5e3ed8b59b None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:49.195318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf648b-3c72-4dad-ad67-dd5e3ed8b59b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:49.199681 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf648b-3c72-4dad-ad67-dd5e3ed8b59b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:49.294639 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391e8d4-3f8f-4751-bbc4-d14b47fc568d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:49.323998 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 371/715] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 18 08:25:48 2022] POST /identity/v3/users => generated 254 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:25:49.325621 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf648b-3c72-4dad-ad67-dd5e3ed8b59b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:49.356497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:49.387187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7523cd-6984-473a-a051-00265f449536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:49.388377 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7523cd-6984-473a-a051-00265f449536 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:49.388881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7523cd-6984-473a-a051-00265f449536 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:49.389373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7523cd-6984-473a-a051-00265f449536 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:49.390982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7523cd-6984-473a-a051-00265f449536 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:49.391516 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7523cd-6984-473a-a051-00265f449536 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:49.394466 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7523cd-6984-473a-a051-00265f449536 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:49.444007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 345/716] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:25:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:49.451661 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7523cd-6984-473a-a051-00265f449536 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:49.548414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:49.619344 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf648b-3c72-4dad-ad67-dd5e3ed8b59b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:49.634497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7523cd-6984-473a-a051-00265f449536 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:49.674499 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23141d07-e2ab-40d0-a2c4-72497d0f1cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:49.675274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23141d07-e2ab-40d0-a2c4-72497d0f1cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:49.675560 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23141d07-e2ab-40d0-a2c4-72497d0f1cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:49.675863 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23141d07-e2ab-40d0-a2c4-72497d0f1cb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:49.676808 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23141d07-e2ab-40d0-a2c4-72497d0f1cb4 None admin] RBAC: Adding query filter params (role.id=467d49b8a0d3435aaba58464cc8ef85c, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=84cba474fc7c470ea119de697b9d7ef9) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:49.677121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23141d07-e2ab-40d0-a2c4-72497d0f1cb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:49.679201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23141d07-e2ab-40d0-a2c4-72497d0f1cb4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:49.685301 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a93ed-b62e-4d3b-8f09-fdf1f1643cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:49.686440 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a93ed-b62e-4d3b-8f09-fdf1f1643cfd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:49.686655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 346/717] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:25:49 2022] GET /identity/v3/role_assignments?user.id=84cba474fc7c470ea119de697b9d7ef9&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=467d49b8a0d3435aaba58464cc8ef85c => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:49.686935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a93ed-b62e-4d3b-8f09-fdf1f1643cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:49.687420 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a93ed-b62e-4d3b-8f09-fdf1f1643cfd None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:49.688847 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7a93ed-b62e-4d3b-8f09-fdf1f1643cfd None admin] RBAC: Adding query filter params (name=network) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:49.689395 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7a93ed-b62e-4d3b-8f09-fdf1f1643cfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:49.694825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7a93ed-b62e-4d3b-8f09-fdf1f1643cfd None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:49.706348 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 372/718] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 18 08:25:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:49.713294 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23141d07-e2ab-40d0-a2c4-72497d0f1cb4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:49.715045 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bab3a-5a93-445f-a1db-5fe7b2b0598f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:49.715343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bab3a-5a93-445f-a1db-5fe7b2b0598f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:49.715627 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bab3a-5a93-445f-a1db-5fe7b2b0598f None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:49.717577 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 373/719] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:49.723498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997047e-b149-4919-bb11-8e743af84c6b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:49.723846 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997047e-b149-4919-bb11-8e743af84c6b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:49.724232 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997047e-b149-4919-bb11-8e743af84c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:49.755748 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8997047e-b149-4919-bb11-8e743af84c6b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:49.759066 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8997047e-b149-4919-bb11-8e743af84c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:49.896829 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:49.897063 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23141d07-e2ab-40d0-a2c4-72497d0f1cb4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:49.900833 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 374/720] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:25:49.901375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:49.944637 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997047e-b149-4919-bb11-8e743af84c6b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:49.969260 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b084745-ccbc-4e38-ba91-3eb63b6351d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:49.969955 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b084745-ccbc-4e38-ba91-3eb63b6351d0 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:49.970238 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b084745-ccbc-4e38-ba91-3eb63b6351d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:49.970515 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b084745-ccbc-4e38-ba91-3eb63b6351d0 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:49.971237 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b084745-ccbc-4e38-ba91-3eb63b6351d0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:49.971542 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b084745-ccbc-4e38-ba91-3eb63b6351d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:49.977996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b084745-ccbc-4e38-ba91-3eb63b6351d0 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:49.987197 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 347/721] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 18 08:25:49 2022] GET /identity/v3/services?type=network => generated 124 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:50.140268 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997047e-b149-4919-bb11-8e743af84c6b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:50.201629 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a44ad7-9731-4c9f-9ff6-b96e933683d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:50.202458 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a44ad7-9731-4c9f-9ff6-b96e933683d0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:50.202743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a44ad7-9731-4c9f-9ff6-b96e933683d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:50.203019 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a44ad7-9731-4c9f-9ff6-b96e933683d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:50.219386 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a44ad7-9731-4c9f-9ff6-b96e933683d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:50.227968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f3a44ad7-9731-4c9f-9ff6-b96e933683d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:25:50.227968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:50.227968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:50.227968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:50.227968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:50.227968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:50.227968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:50.227968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:50.227968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:50.227968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:50.227968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:50.227968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:50.227968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:50.227968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:50.227968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:50.227968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:50.228969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:50.228969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:50.228969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:50.228969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:50.228969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:50.228969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:50.228969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:50.228969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:50.228969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:50.228969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:50.228969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:50.228969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:50.228969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:50.228969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:50.228969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:50.228969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:50.230263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:50.230263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:50.230263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:50.230263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:50.230263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:50.230263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:50.230263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:50.230263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:50.230263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:50.230263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:50.230263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:25:50.230263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:50.230263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 375/722] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:25:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:25:50.235620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b084745-ccbc-4e38-ba91-3eb63b6351d0 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:50.384291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b084745-ccbc-4e38-ba91-3eb63b6351d0 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:50.429294 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9c515-7bd9-4274-a386-7ae49f901902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:50.429294 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9c515-7bd9-4274-a386-7ae49f901902 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:50.429593 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9c515-7bd9-4274-a386-7ae49f901902 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:50.429821 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9c515-7bd9-4274-a386-7ae49f901902 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:50.430540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a9c515-7bd9-4274-a386-7ae49f901902 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:50.430850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a9c515-7bd9-4274-a386-7ae49f901902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:50.434428 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a9c515-7bd9-4274-a386-7ae49f901902 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:50.441890 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 348/723] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:25:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:25:50.448366 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a44ad7-9731-4c9f-9ff6-b96e933683d0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:50.519457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:50.577580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a44ad7-9731-4c9f-9ff6-b96e933683d0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:50.625790 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03693243-be71-45ff-a995-7ea1199ec688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:50.626600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03693243-be71-45ff-a995-7ea1199ec688 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:50.626887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03693243-be71-45ff-a995-7ea1199ec688 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:50.627166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03693243-be71-45ff-a995-7ea1199ec688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:50.639616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03693243-be71-45ff-a995-7ea1199ec688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:50.641150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03693243-be71-45ff-a995-7ea1199ec688 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:50.660077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-03693243-be71-45ff-a995-7ea1199ec688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:50.660077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:50.660077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:50.660077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:50.660077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:50.660077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:50.660077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:50.660077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:50.660077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:50.660077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:50.660077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:50.660077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:50.660077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:50.660077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:50.660077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:50.661042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:50.661042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:50.661042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:50.661042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:50.661042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:50.661042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:50.661042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:50.661042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:50.661042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:50.661042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:50.661042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:50.661042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:50.661042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:50.661042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:50.661042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:50.661042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:50.661042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:50.663563 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:50.663563 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:50.663563 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:50.663563 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:50.663563 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 376/724] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:50.667940 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9c515-7bd9-4274-a386-7ae49f901902 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:50.699032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:50.801611 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9c515-7bd9-4274-a386-7ae49f901902 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:50.858178 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2cacc-64c5-4978-b8d0-e32c9279ddc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:50.858871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2cacc-64c5-4978-b8d0-e32c9279ddc3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:50.859155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2cacc-64c5-4978-b8d0-e32c9279ddc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:50.859431 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2cacc-64c5-4978-b8d0-e32c9279ddc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:50.860200 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2cacc-64c5-4978-b8d0-e32c9279ddc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:50.860506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2cacc-64c5-4978-b8d0-e32c9279ddc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:50.861943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2cacc-64c5-4978-b8d0-e32c9279ddc3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:50.883689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 349/725] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:50.889242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03693243-be71-45ff-a995-7ea1199ec688 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:51.003478 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03693243-be71-45ff-a995-7ea1199ec688 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:51.020541 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:51.044712 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c9c07-c97d-492f-89c6-4dde49f8d010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:51.045770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c9c07-c97d-492f-89c6-4dde49f8d010 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:51.046064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c9c07-c97d-492f-89c6-4dde49f8d010 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:51.046342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c9c07-c97d-492f-89c6-4dde49f8d010 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:51.056147 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c9c07-c97d-492f-89c6-4dde49f8d010 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:51.057561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c9c07-c97d-492f-89c6-4dde49f8d010 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:51.074872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-894c9c07-c97d-492f-89c6-4dde49f8d010 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 08:25:51.074872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:51.074872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:51.074872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:51.074872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:51.074872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:51.074872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:51.074872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:51.074872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:51.074872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:51.074872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:51.074872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:51.074872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:25:51.074872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:25:51.074872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:25:51.074872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:25:51.075949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:51.075949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:51.075949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:25:51.075949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:51.075949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:25:51.075949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:51.075949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:51.075949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:51.075949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:51.075949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:51.075949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:51.075949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:51.075949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:51.075949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:51.075949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:51.075949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:51.075949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:51.075949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:51.077363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:51.077363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:51.077363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:25:51.077363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:25:51.077363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:25:51.077363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:25:51.077363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:25:51.077363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:25:51.077363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 08:25:51.077363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:51.077363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 377/726] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:25:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:51.082344 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2cacc-64c5-4978-b8d0-e32c9279ddc3 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:51.203112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:51.217465 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2cacc-64c5-4978-b8d0-e32c9279ddc3 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:51.230713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2b414-2667-4a9b-b6c8-f7edff386e82 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:51.231039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2b414-2667-4a9b-b6c8-f7edff386e82 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:51.231277 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2b414-2667-4a9b-b6c8-f7edff386e82 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:51.232984 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 378/727] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:25:51.239247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f045385-00c0-4331-8c55-5ae586c55877 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:51.239554 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f045385-00c0-4331-8c55-5ae586c55877 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:51.239869 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f045385-00c0-4331-8c55-5ae586c55877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:51.271889 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0f045385-00c0-4331-8c55-5ae586c55877 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:51.275404 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0f045385-00c0-4331-8c55-5ae586c55877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:51.294502 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ee678-5772-47a2-8356-6a8e26cc0ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:51.296130 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ee678-5772-47a2-8356-6a8e26cc0ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:51.296596 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ee678-5772-47a2-8356-6a8e26cc0ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:51.297075 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ee678-5772-47a2-8356-6a8e26cc0ec7 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:51.298120 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ee678-5772-47a2-8356-6a8e26cc0ec7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:51.298616 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ee678-5772-47a2-8356-6a8e26cc0ec7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:51.302983 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ee678-5772-47a2-8356-6a8e26cc0ec7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:51.348243 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 350/728] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Jul 18 08:25:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:51.353929 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ee678-5772-47a2-8356-6a8e26cc0ec7 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:51.388264 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 379/729] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:25:51.435697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f045385-00c0-4331-8c55-5ae586c55877 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:51.555067 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f045385-00c0-4331-8c55-5ae586c55877 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:51.572110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:51.595396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ed5fb-aa85-4ec0-8074-8be25a3590b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:51.596495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ed5fb-aa85-4ec0-8074-8be25a3590b4 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:51.596965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ed5fb-aa85-4ec0-8074-8be25a3590b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:51.597423 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ed5fb-aa85-4ec0-8074-8be25a3590b4 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:51.598464 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ed5fb-aa85-4ec0-8074-8be25a3590b4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:51.600813 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ed5fb-aa85-4ec0-8074-8be25a3590b4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:51.625567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ee678-5772-47a2-8356-6a8e26cc0ec7 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:51.669321 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab869b13-6c6b-4ebb-be7d-a98669bc1483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:51.670332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab869b13-6c6b-4ebb-be7d-a98669bc1483 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:51.670851 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab869b13-6c6b-4ebb-be7d-a98669bc1483 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:51.671337 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab869b13-6c6b-4ebb-be7d-a98669bc1483 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:51.698138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 380/730] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 18 08:25:51 2022] POST /identity/v3/services => generated 251 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:25:51.702113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:51.790826 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab869b13-6c6b-4ebb-be7d-a98669bc1483 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:51.792757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab869b13-6c6b-4ebb-be7d-a98669bc1483 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:51.956189 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 351/731] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:25:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 08:25:51.965921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ed5fb-aa85-4ec0-8074-8be25a3590b4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:52.128505 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ed5fb-aa85-4ec0-8074-8be25a3590b4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:52.184308 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fdad9-38e4-47b6-91dd-181530aa2a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:52.185134 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fdad9-38e4-47b6-91dd-181530aa2a73 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:52.185412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fdad9-38e4-47b6-91dd-181530aa2a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:52.185691 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fdad9-38e4-47b6-91dd-181530aa2a73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:52.192763 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2af95-f862-4be6-ab92-d77095896f40 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:52.194315 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2af95-f862-4be6-ab92-d77095896f40 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:52.195402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2af95-f862-4be6-ab92-d77095896f40 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:52.198293 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 352/732] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:52.213271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671e33a-1dfe-4d16-816b-2aeb8673e8c0 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:52.213271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671e33a-1dfe-4d16-816b-2aeb8673e8c0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:52.213271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671e33a-1dfe-4d16-816b-2aeb8673e8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:52.217543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832fdad9-38e4-47b6-91dd-181530aa2a73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:52.220822 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832fdad9-38e4-47b6-91dd-181530aa2a73 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:52.257707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-832fdad9-38e4-47b6-91dd-181530aa2a73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:52.257707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:52.257707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:52.257707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:52.257707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:52.257707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:52.257707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:52.257707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:52.257707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:52.257707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:52.257707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:52.257707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:52.257707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:52.257707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:52.257707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:52.259492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:52.259492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:52.259492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:52.259492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:52.259492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:52.259492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:52.259492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:52.259492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:52.259492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:52.259492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:52.259492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:52.259492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:52.259492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:52.259492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:52.259492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:52.259492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:52.259492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:52.261251 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:52.261251 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:52.261251 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:52.261251 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:52.261694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0671e33a-1dfe-4d16-816b-2aeb8673e8c0 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:52.261694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0671e33a-1dfe-4d16-816b-2aeb8673e8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:52.265741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 381/733] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:52.277087 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fdad9-38e4-47b6-91dd-181530aa2a73 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:52.323558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:52.433059 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:52.525152 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 353/734] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:52 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 08:25:52.564437 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671e33a-1dfe-4d16-816b-2aeb8673e8c0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:52.628774 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fdad9-38e4-47b6-91dd-181530aa2a73 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:52.769916 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c6b54-cc48-41c7-a0dd-04f7c31538af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:52.770869 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c6b54-cc48-41c7-a0dd-04f7c31538af None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:52.771887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c6b54-cc48-41c7-a0dd-04f7c31538af None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:52.772358 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c6b54-cc48-41c7-a0dd-04f7c31538af None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:52.773590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c6b54-cc48-41c7-a0dd-04f7c31538af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:52.774042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c6b54-cc48-41c7-a0dd-04f7c31538af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:52.776209 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c6b54-cc48-41c7-a0dd-04f7c31538af None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:52.805576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671e33a-1dfe-4d16-816b-2aeb8673e8c0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:52.820729 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 382/735] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 545 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:25:52.830096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c6b54-cc48-41c7-a0dd-04f7c31538af None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:52.882003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb30af-0907-4a2c-8775-abf4a4eafce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:52.883382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb30af-0907-4a2c-8775-abf4a4eafce1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:52.883908 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb30af-0907-4a2c-8775-abf4a4eafce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:52.884399 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb30af-0907-4a2c-8775-abf4a4eafce1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:52.905510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb30af-0907-4a2c-8775-abf4a4eafce1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:52.921138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-d4bb30af-0907-4a2c-8775-abf4a4eafce1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:25:52.921138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:52.921138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:52.921138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:52.921138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:52.921138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:52.921138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:52.921138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:52.921138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:52.921138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:52.921138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:52.921138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:52.921138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:52.921138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:52.921138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:52.921138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:52.922151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:52.922151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:52.922151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:52.922151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:52.922151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:52.922151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:52.922151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:52.922151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:52.922151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:52.922151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:52.922151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:52.922151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:52.922151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:52.922151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:52.922151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:52.922151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:52.923093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:52.923093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:52.923093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:52.923093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:52.923093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:52.923093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:52.923093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:52.923093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:52.923093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:52.923093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:52.923093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:25:52.923093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:52.924331 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 354/736] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:25:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:25:52.925871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bdfe4-5279-44e2-a4ce-898e7456caf6 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:52.926366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bdfe4-5279-44e2-a4ce-898e7456caf6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:52.926846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bdfe4-5279-44e2-a4ce-898e7456caf6 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:52.929852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 355/737] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:52.936061 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c58592-8903-420e-8c28-1391b4b927d4 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:52.936627 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c58592-8903-420e-8c28-1391b4b927d4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:52.936918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c58592-8903-420e-8c28-1391b4b927d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:52.978734 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-e7c58592-8903-420e-8c28-1391b4b927d4 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:52.982404 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e7c58592-8903-420e-8c28-1391b4b927d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:53.000975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:53.023871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:53.056928 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c6b54-cc48-41c7-a0dd-04f7c31538af None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:53.093697 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 356/738] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:52 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 08:25:53.095193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c58592-8903-420e-8c28-1391b4b927d4 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:53.107963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f2b503-9a36-4cb1-9aec-fb24eeb3d875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:53.108887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2b503-9a36-4cb1-9aec-fb24eeb3d875 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:53.109292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2b503-9a36-4cb1-9aec-fb24eeb3d875 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:53.109574 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2b503-9a36-4cb1-9aec-fb24eeb3d875 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:53.121592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f2b503-9a36-4cb1-9aec-fb24eeb3d875 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:53.123190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f2b503-9a36-4cb1-9aec-fb24eeb3d875 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:53.138524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-03f2b503-9a36-4cb1-9aec-fb24eeb3d875 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:53.138524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:53.138524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:53.138524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:53.138524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:53.138524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:53.138524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:53.138524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:53.138524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:53.138524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:53.138524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:53.138524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:53.138524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:25:53.138524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:25:53.138524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:25:53.140393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:25:53.140393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:53.140393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:53.140393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:53.140393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:53.140393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:53.140393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:53.140393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:53.140393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:53.140393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:53.140393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:53.140393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:53.140393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:53.140393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:53.140393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:53.140393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:53.140393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:53.142141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:25:53.142141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:25:53.142141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:25:53.142141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:25:53.142141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:25:53.142141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:25:53.142141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:53.142141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:53.142141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 383/739] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:53.142141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4dd81-bc54-4aef-93c3-d78cfa89c591 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:53.142141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4dd81-bc54-4aef-93c3-d78cfa89c591 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:53.143271 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4dd81-bc54-4aef-93c3-d78cfa89c591 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:53.144166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 384/740] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:53.144166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4dd81-bc54-4aef-93c3-d78cfa89c591 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:53.270239 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c58592-8903-420e-8c28-1391b4b927d4 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:53.313919 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4dd81-bc54-4aef-93c3-d78cfa89c591 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:53.342064 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ca1ce-ff6c-42ed-b321-f2d99c3da8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:53.343221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ca1ce-ff6c-42ed-b321-f2d99c3da8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:53.343764 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ca1ce-ff6c-42ed-b321-f2d99c3da8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:53.344232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ca1ce-ff6c-42ed-b321-f2d99c3da8b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:53.345502 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ca1ce-ff6c-42ed-b321-f2d99c3da8b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:53.345991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ca1ce-ff6c-42ed-b321-f2d99c3da8b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:53.348436 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ca1ce-ff6c-42ed-b321-f2d99c3da8b9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:53.355603 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:53.366922 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 357/741] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:25:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:53.368476 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b08823-e7ed-4bac-95f8-662389a84f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:53.369482 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b08823-e7ed-4bac-95f8-662389a84f76 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:53.370105 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982140f-3f25-4f8f-9d8a-d19890a85b7d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:53.370621 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982140f-3f25-4f8f-9d8a-d19890a85b7d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:53.375577 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982140f-3f25-4f8f-9d8a-d19890a85b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:53.375827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b08823-e7ed-4bac-95f8-662389a84f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:53.375827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b08823-e7ed-4bac-95f8-662389a84f76 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:53.375827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b08823-e7ed-4bac-95f8-662389a84f76 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:53.375827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b08823-e7ed-4bac-95f8-662389a84f76 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:53.401483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-e982140f-3f25-4f8f-9d8a-d19890a85b7d None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:53.403630 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e982140f-3f25-4f8f-9d8a-d19890a85b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:53.442620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:53.487707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 385/742] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 18 08:25:53 2022] POST /identity/v3/services => generated 240 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:25:53.487707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b08823-e7ed-4bac-95f8-662389a84f76 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:53.555997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 358/743] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:25:53.557557 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982140f-3f25-4f8f-9d8a-d19890a85b7d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:53.754165 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b08823-e7ed-4bac-95f8-662389a84f76 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:53.804697 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982140f-3f25-4f8f-9d8a-d19890a85b7d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:53.838194 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b44f20-6128-4169-b6b3-6fbc8cf81e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:53.838194 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b44f20-6128-4169-b6b3-6fbc8cf81e49 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:53.838194 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b44f20-6128-4169-b6b3-6fbc8cf81e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:53.838194 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b44f20-6128-4169-b6b3-6fbc8cf81e49 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:53.838829 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b44f20-6128-4169-b6b3-6fbc8cf81e49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:53.838829 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b44f20-6128-4169-b6b3-6fbc8cf81e49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:53.839434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b44f20-6128-4169-b6b3-6fbc8cf81e49 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:53.862219 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 386/744] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:25:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:53.862219 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b44f20-6128-4169-b6b3-6fbc8cf81e49 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:53.875852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a9400-f781-4c65-b27e-a00856f10af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:53.878043 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a9400-f781-4c65-b27e-a00856f10af1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:53.878547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a9400-f781-4c65-b27e-a00856f10af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:53.879259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a9400-f781-4c65-b27e-a00856f10af1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:53.906957 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a9400-f781-4c65-b27e-a00856f10af1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:53.909878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a9400-f781-4c65-b27e-a00856f10af1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:53.921461 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:53.949318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-e20a9400-f781-4c65-b27e-a00856f10af1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:53.949318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:53.949318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:53.949318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:53.949318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:53.949318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:53.949318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:53.949318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:53.949318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:53.949318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:53.949318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:53.949318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:53.949318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:53.949318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:53.949318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:53.951074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:53.951074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:53.951074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:53.951074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:53.951074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:53.951074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:53.951074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:53.951074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:53.951074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:53.951074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:53.951074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:53.951074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:53.951074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:53.951074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:53.951074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:53.951074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:53.951074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:53.953616 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:53.953616 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:53.953616 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:53.953616 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:53.953616 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 359/745] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:53.955818 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a9400-f781-4c65-b27e-a00856f10af1 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:53.987501 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b44f20-6128-4169-b6b3-6fbc8cf81e49 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:54.000250 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:54.067706 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb5b50-810e-4ba1-b4ce-32a72452c394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:54.070483 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb5b50-810e-4ba1-b4ce-32a72452c394 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:54.071496 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb5b50-810e-4ba1-b4ce-32a72452c394 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:54.072033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb5b50-810e-4ba1-b4ce-32a72452c394 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:54.101870 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb5b50-810e-4ba1-b4ce-32a72452c394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:54.110307 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a9400-f781-4c65-b27e-a00856f10af1 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:54.118590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-cbeb5b50-810e-4ba1-b4ce-32a72452c394 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:54.118590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:54.118590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:54.118590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:54.118590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:54.118590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:54.118590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:54.118590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:54.118590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:54.118590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:54.118590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:54.118590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:54.118590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:54.118590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:54.118590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:54.118590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:54.119600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:54.119600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:54.119600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:54.119600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:54.119600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:54.119600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:54.119600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:54.119600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:54.119600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:54.119600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:54.119600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:54.119600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:54.119600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:54.119600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:54.119600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:54.119600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:54.120466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:54.120466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:54.120466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:54.120466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:54.120466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:54.120466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:54.120466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:54.120466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:54.120466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:54.120466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:54.120466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:25:54.120466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:54.128207 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 387/746] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:25:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:54.129925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb5b50-810e-4ba1-b4ce-32a72452c394 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:54.155761 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2403f30-ce68-4671-ae2e-64cf5a36ec50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:54.156798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2403f30-ce68-4671-ae2e-64cf5a36ec50 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:54.157280 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2403f30-ce68-4671-ae2e-64cf5a36ec50 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:54.157771 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2403f30-ce68-4671-ae2e-64cf5a36ec50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:54.158836 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2403f30-ce68-4671-ae2e-64cf5a36ec50 None admin] RBAC: Adding query filter params (role.id=467d49b8a0d3435aaba58464cc8ef85c, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=09a5c2e804504038817cad76544dce5f) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:54.159352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2403f30-ce68-4671-ae2e-64cf5a36ec50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:54.161072 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2403f30-ce68-4671-ae2e-64cf5a36ec50 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:54.168971 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 360/747] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:25:53 2022] GET /identity/v3/role_assignments?user.id=09a5c2e804504038817cad76544dce5f&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=467d49b8a0d3435aaba58464cc8ef85c => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:54.175708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2403f30-ce68-4671-ae2e-64cf5a36ec50 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:54.310929 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:54.349095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb5b50-810e-4ba1-b4ce-32a72452c394 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:54.351281 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2403f30-ce68-4671-ae2e-64cf5a36ec50 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:54.396400 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:54.397573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061bf9d-8c70-4fc6-9ea1-7b47665ccd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:54.398269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061bf9d-8c70-4fc6-9ea1-7b47665ccd22 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:54.398548 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061bf9d-8c70-4fc6-9ea1-7b47665ccd22 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:54.399420 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061bf9d-8c70-4fc6-9ea1-7b47665ccd22 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:54.400233 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061bf9d-8c70-4fc6-9ea1-7b47665ccd22 None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:54.400556 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061bf9d-8c70-4fc6-9ea1-7b47665ccd22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:54.401932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061bf9d-8c70-4fc6-9ea1-7b47665ccd22 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:54.405456 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b426b-e206-4c91-8138-f16ccd838510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:54.406437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b426b-e206-4c91-8138-f16ccd838510 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:54.406871 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b426b-e206-4c91-8138-f16ccd838510 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:54.407304 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b426b-e206-4c91-8138-f16ccd838510 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:54.408261 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b426b-e206-4c91-8138-f16ccd838510 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:54.408759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b426b-e206-4c91-8138-f16ccd838510 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:54.411381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b426b-e206-4c91-8138-f16ccd838510 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:54.413994 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 361/748] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:25:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:54.423728 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e84903-6db1-47bf-9e39-e57b79173df1 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:54.423728 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e84903-6db1-47bf-9e39-e57b79173df1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:54.423728 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e84903-6db1-47bf-9e39-e57b79173df1 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:54.423728 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 362/749] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:54.424126 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e84903-6db1-47bf-9e39-e57b79173df1 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:54.444358 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 388/750] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:54.446594 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee2118-1424-4b5b-b7d7-4e2bb1ebc51f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:54.447023 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee2118-1424-4b5b-b7d7-4e2bb1ebc51f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:54.447413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee2118-1424-4b5b-b7d7-4e2bb1ebc51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:54.476495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c5ee2118-1424-4b5b-b7d7-4e2bb1ebc51f None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:54.478623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c5ee2118-1424-4b5b-b7d7-4e2bb1ebc51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:54.562760 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e84903-6db1-47bf-9e39-e57b79173df1 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:54.601928 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:54.609039 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f38675-9413-4a3c-ab45-79475f8cd976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:54.610040 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f38675-9413-4a3c-ab45-79475f8cd976 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:54.610381 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f38675-9413-4a3c-ab45-79475f8cd976 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:54.610833 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f38675-9413-4a3c-ab45-79475f8cd976 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:54.615231 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 389/751] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:25:54.615231 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee2118-1424-4b5b-b7d7-4e2bb1ebc51f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:54.639278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f38675-9413-4a3c-ab45-79475f8cd976 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:54.641446 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f38675-9413-4a3c-ab45-79475f8cd976 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:54.672924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-95f38675-9413-4a3c-ab45-79475f8cd976 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:54.672924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:54.672924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:54.672924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:54.672924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:54.672924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:54.672924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:54.672924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:54.672924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:54.672924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:54.672924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:54.672924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:54.672924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:54.672924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:54.672924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:54.673879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:54.673879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:54.673879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:54.673879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:54.673879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:54.673879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:54.673879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:54.673879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:54.673879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:54.673879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:54.673879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:54.673879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:54.673879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:54.673879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:54.673879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:54.673879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:54.673879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:54.674767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:54.674767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:54.674767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:54.674767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:54.674767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 363/752] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:54.675487 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f38675-9413-4a3c-ab45-79475f8cd976 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:54.817527 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f38675-9413-4a3c-ab45-79475f8cd976 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:54.862654 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:54.868461 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109791f7-199b-4ea8-a3cb-afed6705a155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:54.869581 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109791f7-199b-4ea8-a3cb-afed6705a155 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:54.869712 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee2118-1424-4b5b-b7d7-4e2bb1ebc51f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:54.870135 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109791f7-199b-4ea8-a3cb-afed6705a155 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:54.870567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109791f7-199b-4ea8-a3cb-afed6705a155 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:54.871374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109791f7-199b-4ea8-a3cb-afed6705a155 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:54.873017 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109791f7-199b-4ea8-a3cb-afed6705a155 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:54.887812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-109791f7-199b-4ea8-a3cb-afed6705a155 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 08:25:54.887812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:54.887812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:25:54.887812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:25:54.887812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:25:54.887812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:25:54.887812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:25:54.887812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:25:54.887812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 08:25:54.887812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:54.887812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:25:54.887812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:54.887812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:54.887812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:54.887812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:54.887812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:54.889194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:54.889194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:54.889194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:54.889194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:54.889194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:54.889194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:54.889194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:54.889194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:25:54.889194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:25:54.889194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:25:54.889194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:25:54.889194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:54.889194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:54.889194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:54.889194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:54.889194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:54.890024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:54.890024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:54.890024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:54.890024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:54.890024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:54.890024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:54.890024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:54.890024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:54.890024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:54.890024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:54.890024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:54.890024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:25:54.890024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:25:54.890024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 08:25:54.890024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:54.892793 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 364/753] 10.222.0.57 () {56 vars in 1129 bytes} [Mon Jul 18 08:25:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:54.893904 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109791f7-199b-4ea8-a3cb-afed6705a155 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:54.956126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ececd-18da-406e-b673-12ce44cd2c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:54.956973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ececd-18da-406e-b673-12ce44cd2c1b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:54.957250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ececd-18da-406e-b673-12ce44cd2c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:54.957523 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ececd-18da-406e-b673-12ce44cd2c1b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:54.961732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:54.978025 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ececd-18da-406e-b673-12ce44cd2c1b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:54.982477 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ececd-18da-406e-b673-12ce44cd2c1b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:55.026040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-885ececd-18da-406e-b673-12ce44cd2c1b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 08:25:55.026040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:55.026040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:55.026040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:55.026040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:55.026040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:55.026040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:55.026040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:55.026040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:55.026040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:55.026040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:55.026040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:55.026040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:25:55.026040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:25:55.026040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:25:55.026040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:25:55.027827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:55.027827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:55.027827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:25:55.027827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:55.027827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:25:55.027827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:55.027827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:55.027827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:55.027827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:55.027827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:55.027827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:55.027827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:55.027827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:55.027827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:55.027827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:55.027827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:55.027827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:55.027827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:55.029671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:55.029671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:55.029671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:25:55.029671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:25:55.029671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:25:55.029671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:25:55.029671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:25:55.029671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:25:55.029671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 08:25:55.029671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:55.032499 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109791f7-199b-4ea8-a3cb-afed6705a155 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:55.036109 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 390/754] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 18 08:25:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:55.036109 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ececd-18da-406e-b673-12ce44cd2c1b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:55.076320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f43978-de61-42f1-a8eb-03136ae0c440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:55.076974 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f43978-de61-42f1-a8eb-03136ae0c440 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:55.077279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f43978-de61-42f1-a8eb-03136ae0c440 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:55.077521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f43978-de61-42f1-a8eb-03136ae0c440 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:55.078251 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f43978-de61-42f1-a8eb-03136ae0c440 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:55.078561 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f43978-de61-42f1-a8eb-03136ae0c440 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:55.080105 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f43978-de61-42f1-a8eb-03136ae0c440 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:55.102618 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 365/755] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:55.102618 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f43978-de61-42f1-a8eb-03136ae0c440 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:55.213093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f43978-de61-42f1-a8eb-03136ae0c440 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:55.224527 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:55.251022 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ececd-18da-406e-b673-12ce44cd2c1b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:55.253762 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5b7b0-3950-4ff8-b1eb-a44cee0423f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:55.254482 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5b7b0-3950-4ff8-b1eb-a44cee0423f3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:55.254802 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5b7b0-3950-4ff8-b1eb-a44cee0423f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:55.255077 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5b7b0-3950-4ff8-b1eb-a44cee0423f3 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:55.257070 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5b7b0-3950-4ff8-b1eb-a44cee0423f3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:55.257330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5b7b0-3950-4ff8-b1eb-a44cee0423f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:55.258734 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5b7b0-3950-4ff8-b1eb-a44cee0423f3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:55.284428 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 366/756] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 18 08:25:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:55.285434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5b7b0-3950-4ff8-b1eb-a44cee0423f3 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:55.325240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f9f9b-7685-4191-959a-e98dfe943716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:55.326338 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f9f9b-7685-4191-959a-e98dfe943716 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:55.326838 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f9f9b-7685-4191-959a-e98dfe943716 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:55.327316 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f9f9b-7685-4191-959a-e98dfe943716 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:55.328719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f9f9b-7685-4191-959a-e98dfe943716 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:55.329263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f9f9b-7685-4191-959a-e98dfe943716 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:55.331884 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f9f9b-7685-4191-959a-e98dfe943716 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:55.341582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 391/757] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Jul 18 08:25:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:55.343021 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f9f9b-7685-4191-959a-e98dfe943716 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:55.391881 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:55.403527 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5b7b0-3950-4ff8-b1eb-a44cee0423f3 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:55.445740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd79cf-8c57-4bc1-9408-45e3b54d56eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:55.446578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd79cf-8c57-4bc1-9408-45e3b54d56eb None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:55.446857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd79cf-8c57-4bc1-9408-45e3b54d56eb None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:55.447122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd79cf-8c57-4bc1-9408-45e3b54d56eb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:55.456017 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd79cf-8c57-4bc1-9408-45e3b54d56eb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:55.457269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd79cf-8c57-4bc1-9408-45e3b54d56eb None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:55.476680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-85dd79cf-8c57-4bc1-9408-45e3b54d56eb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 08:25:55.476680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:55.476680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:55.476680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:55.476680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:55.476680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:55.476680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:55.476680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:55.476680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:55.476680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:55.476680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:55.476680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:55.476680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:25:55.476680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:25:55.476680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:25:55.476680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:25:55.478537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:55.478537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:55.478537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:25:55.478537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:55.478537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:25:55.478537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:55.478537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:55.478537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:55.478537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:55.478537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:55.478537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:55.478537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:55.478537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:55.478537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:55.478537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:55.478537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:55.478537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:55.478537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:55.488426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:55.488426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:55.488426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:25:55.488426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:25:55.488426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:25:55.488426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:25:55.488426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:25:55.488426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:25:55.488426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 08:25:55.488426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:55.488426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 367/758] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 18 08:25:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:55.488426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd79cf-8c57-4bc1-9408-45e3b54d56eb None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:55.512015 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:55.556692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f9f9b-7685-4191-959a-e98dfe943716 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:55.601958 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd79cf-8c57-4bc1-9408-45e3b54d56eb None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:55.625396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82e5d9-4418-49e7-aa22-2b24e8dd1f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:55.625396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82e5d9-4418-49e7-aa22-2b24e8dd1f84 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:55.628393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82e5d9-4418-49e7-aa22-2b24e8dd1f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:55.628393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82e5d9-4418-49e7-aa22-2b24e8dd1f84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:55.647857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ebeef-02b9-4425-a6c8-44590a8c7444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:55.648543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ebeef-02b9-4425-a6c8-44590a8c7444 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:55.648824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ebeef-02b9-4425-a6c8-44590a8c7444 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:55.649095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ebeef-02b9-4425-a6c8-44590a8c7444 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:55.649824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ebeef-02b9-4425-a6c8-44590a8c7444 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:55.650129 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ebeef-02b9-4425-a6c8-44590a8c7444 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:55.651704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ebeef-02b9-4425-a6c8-44590a8c7444 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:55.658228 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 368/759] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Jul 18 08:25:55 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:55.659157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ebeef-02b9-4425-a6c8-44590a8c7444 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:55.660471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82e5d9-4418-49e7-aa22-2b24e8dd1f84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:55.666009 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82e5d9-4418-49e7-aa22-2b24e8dd1f84 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:55.717534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-4a82e5d9-4418-49e7-aa22-2b24e8dd1f84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:55.717534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:55.717534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:55.717534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:55.717534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:55.717534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:55.717534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:55.717534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:55.717534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:55.717534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:55.717534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:55.717534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:55.717534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:55.717534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:55.717534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:55.719511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:55.719511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:55.719511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:55.719511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:55.719511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:55.719511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:55.719511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:55.719511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:55.719511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:55.719511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:55.719511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:55.719511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:55.719511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:55.719511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:55.719511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:55.719511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:55.719511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:55.721288 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:55.721288 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:55.721288 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:55.721288 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:55.721288 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 392/760] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:55.723708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82e5d9-4418-49e7-aa22-2b24e8dd1f84 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:55.852886 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:55.875593 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ebeef-02b9-4425-a6c8-44590a8c7444 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:55.904793 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:55.919760 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32215769-fe3e-4043-945f-ea04547b7d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:55.920446 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32215769-fe3e-4043-945f-ea04547b7d9d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:55.920739 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32215769-fe3e-4043-945f-ea04547b7d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:55.921010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32215769-fe3e-4043-945f-ea04547b7d9d None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:55.921840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32215769-fe3e-4043-945f-ea04547b7d9d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:55.922149 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32215769-fe3e-4043-945f-ea04547b7d9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:55.923657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32215769-fe3e-4043-945f-ea04547b7d9d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:55.960535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 369/761] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Jul 18 08:25:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:55.960535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32215769-fe3e-4043-945f-ea04547b7d9d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:55.961935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82e5d9-4418-49e7-aa22-2b24e8dd1f84 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:56.012629 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079939cf-c42b-40e6-8f5f-4cb6f13b0bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:56.013301 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079939cf-c42b-40e6-8f5f-4cb6f13b0bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:56.013597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079939cf-c42b-40e6-8f5f-4cb6f13b0bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:56.013874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079939cf-c42b-40e6-8f5f-4cb6f13b0bb6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:56.014678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079939cf-c42b-40e6-8f5f-4cb6f13b0bb6 None admin] RBAC: Adding query filter params (interface=public, service_id=657e71d79c0547b3adda2d9007f448e7, region_id=RegionOne) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:56.014989 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079939cf-c42b-40e6-8f5f-4cb6f13b0bb6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:56.016437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079939cf-c42b-40e6-8f5f-4cb6f13b0bb6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:56.026363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 393/762] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 18 08:25:55 2022] GET /identity/v3/endpoints?service_id=657e71d79c0547b3adda2d9007f448e7&interface=public®ion_id=RegionOne => generated 194 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:56.027352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079939cf-c42b-40e6-8f5f-4cb6f13b0bb6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:56.154241 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32215769-fe3e-4043-945f-ea04547b7d9d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:56.199785 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cf745-c6ce-4013-9fa6-1ba479f46135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:56.200571 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cf745-c6ce-4013-9fa6-1ba479f46135 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:56.200870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cf745-c6ce-4013-9fa6-1ba479f46135 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:56.201125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cf745-c6ce-4013-9fa6-1ba479f46135 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:56.201848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413cf745-c6ce-4013-9fa6-1ba479f46135 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:56.202153 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413cf745-c6ce-4013-9fa6-1ba479f46135 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:56.203630 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413cf745-c6ce-4013-9fa6-1ba479f46135 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:56.223243 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 370/763] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:56.224878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cf745-c6ce-4013-9fa6-1ba479f46135 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:56.236120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079939cf-c42b-40e6-8f5f-4cb6f13b0bb6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:56.312423 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f75e1-8853-42a7-8bbd-10edd1077c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:56.314916 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f75e1-8853-42a7-8bbd-10edd1077c3a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:56.315770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f75e1-8853-42a7-8bbd-10edd1077c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:56.316668 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f75e1-8853-42a7-8bbd-10edd1077c3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:56.341195 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:56.369824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:56.450648 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f75e1-8853-42a7-8bbd-10edd1077c3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:56.451925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f75e1-8853-42a7-8bbd-10edd1077c3a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:56.535910 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cf745-c6ce-4013-9fa6-1ba479f46135 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:56.595519 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005fc3b-ca4b-4756-9c4e-14f3a683dd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:56.596785 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005fc3b-ca4b-4756-9c4e-14f3a683dd82 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:56.597293 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005fc3b-ca4b-4756-9c4e-14f3a683dd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:56.597795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005fc3b-ca4b-4756-9c4e-14f3a683dd82 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:56.599775 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2005fc3b-ca4b-4756-9c4e-14f3a683dd82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:56.602413 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2005fc3b-ca4b-4756-9c4e-14f3a683dd82 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:56.614714 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 394/764] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:25:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 08:25:56.614714 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f75e1-8853-42a7-8bbd-10edd1077c3a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:56.628336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 371/765] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 18 08:25:56 2022] GET /identity/v3/services => generated 772 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:56.628336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75831988-52ec-4265-88df-c56423b32479 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:56.628336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75831988-52ec-4265-88df-c56423b32479 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:56.628336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75831988-52ec-4265-88df-c56423b32479 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:56.631461 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 372/766] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:56.632985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75831988-52ec-4265-88df-c56423b32479 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:56.678355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:56.758704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:56.788938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f75e1-8853-42a7-8bbd-10edd1077c3a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:56.813226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75831988-52ec-4265-88df-c56423b32479 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:56.848585 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56f49f-2705-4e5c-8007-1d3c54b2c4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:56.850706 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56f49f-2705-4e5c-8007-1d3c54b2c4ea None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:56.850706 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56f49f-2705-4e5c-8007-1d3c54b2c4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:56.850706 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56f49f-2705-4e5c-8007-1d3c54b2c4ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:56.862977 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af56f49f-2705-4e5c-8007-1d3c54b2c4ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:56.866228 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af56f49f-2705-4e5c-8007-1d3c54b2c4ea None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:56.871213 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc699e-0157-4677-b0a6-c8d5ed929f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:56.872482 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc699e-0157-4677-b0a6-c8d5ed929f1f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:56.872966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc699e-0157-4677-b0a6-c8d5ed929f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:56.873440 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc699e-0157-4677-b0a6-c8d5ed929f1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:56.887344 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc699e-0157-4677-b0a6-c8d5ed929f1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:56.889208 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc699e-0157-4677-b0a6-c8d5ed929f1f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:56.892390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-af56f49f-2705-4e5c-8007-1d3c54b2c4ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:56.892390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:56.892390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:56.892390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:56.892390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:56.892390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:56.892390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:56.892390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:56.892390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:56.892390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:56.892390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:56.892390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:56.892390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:25:56.892390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:25:56.892390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:25:56.893309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:25:56.893309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:56.893309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:56.893309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:56.893309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:56.893309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:56.893309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:56.893309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:56.893309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:56.893309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:56.893309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:56.893309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:56.893309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:56.893309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:56.893309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:56.893309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:56.893309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:56.894242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:25:56.894242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:25:56.894242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:25:56.894242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:25:56.894242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:25:56.894242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:25:56.894242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:56.894242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:56.894687 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 395/767] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:56.897509 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6fe49-6b27-4eef-8eaa-41677416c2b4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:56.898028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6fe49-6b27-4eef-8eaa-41677416c2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:56.898523 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6fe49-6b27-4eef-8eaa-41677416c2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:56.915732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-d7cc699e-0157-4677-b0a6-c8d5ed929f1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:56.915732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:56.915732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:56.915732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:56.915732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:56.915732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:56.915732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:56.915732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:56.915732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:56.915732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:56.915732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:56.915732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:56.915732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:56.915732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:56.915732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:56.916783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:56.916783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:56.916783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:56.916783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:56.916783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:56.916783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:56.916783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:56.916783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:56.916783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:56.916783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:56.916783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:56.916783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:56.916783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:56.916783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:56.916783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:56.916783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:56.916783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:56.917682 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:56.917682 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:56.917682 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:56.917682 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:56.919122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 373/768] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:56.920513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459b88b-371f-47da-b917-b89099e76a0c None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:56.920878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459b88b-371f-47da-b917-b89099e76a0c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:56.921160 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459b88b-371f-47da-b917-b89099e76a0c None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:56.922709 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 374/769] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:56.923289 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d459b88b-371f-47da-b917-b89099e76a0c None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:56.933364 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-01a6fe49-6b27-4eef-8eaa-41677416c2b4 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:56.934471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-01a6fe49-6b27-4eef-8eaa-41677416c2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:57.041287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 396/770] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:25:57.043246 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6fe49-6b27-4eef-8eaa-41677416c2b4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:57.100283 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:57.102506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d459b88b-371f-47da-b917-b89099e76a0c None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:57.129988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:57.155544 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f57490-7921-4df4-910c-99388e89a81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:57.156279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f57490-7921-4df4-910c-99388e89a81d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:57.156562 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f57490-7921-4df4-910c-99388e89a81d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:57.156838 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f57490-7921-4df4-910c-99388e89a81d None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:57.157843 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f57490-7921-4df4-910c-99388e89a81d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:57.158167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f57490-7921-4df4-910c-99388e89a81d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:57.159770 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f57490-7921-4df4-910c-99388e89a81d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:57.176354 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 375/771] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:25:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:57.177786 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1708370-f30a-4ac0-8462-ea48df701db2 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:57.178687 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1708370-f30a-4ac0-8462-ea48df701db2 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:57.178916 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1708370-f30a-4ac0-8462-ea48df701db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:57.205100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6fe49-6b27-4eef-8eaa-41677416c2b4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:57.207370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-f1708370-f30a-4ac0-8462-ea48df701db2 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:57.210024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f1708370-f30a-4ac0-8462-ea48df701db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:57.249191 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a964ac-8d30-480d-953b-d521e1932c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:57.249879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a964ac-8d30-480d-953b-d521e1932c1c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:57.250163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a964ac-8d30-480d-953b-d521e1932c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:57.250437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a964ac-8d30-480d-953b-d521e1932c1c None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:57.251233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a964ac-8d30-480d-953b-d521e1932c1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:57.251551 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a964ac-8d30-480d-953b-d521e1932c1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:57.253076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a964ac-8d30-480d-953b-d521e1932c1c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:57.268307 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 397/772] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:57.269204 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a964ac-8d30-480d-953b-d521e1932c1c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:57.331563 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 376/773] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:25:57.333537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1708370-f30a-4ac0-8462-ea48df701db2 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:57.470378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:57.486279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1708370-f30a-4ac0-8462-ea48df701db2 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:57.486612 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a964ac-8d30-480d-953b-d521e1932c1c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:57.506980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:57.533326 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aedf8e-2b40-424f-be51-cf12dcdb7fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:57.534248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aedf8e-2b40-424f-be51-cf12dcdb7fd7 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:57.534533 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aedf8e-2b40-424f-be51-cf12dcdb7fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:57.534817 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aedf8e-2b40-424f-be51-cf12dcdb7fd7 None admin] PATH_INFO: `/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/84cba474fc7c470ea119de697b9d7ef9/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:57.538162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aedf8e-2b40-424f-be51-cf12dcdb7fd7 None admin] RBAC: Authorizing `identity:create_grant(project_id=35c846931fd443fbb83a6fbe0aeaa866, user_id=84cba474fc7c470ea119de697b9d7ef9, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:57.540065 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aedf8e-2b40-424f-be51-cf12dcdb7fd7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:57.552999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d82d1c-4477-4d72-98c3-7fb897db2842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:57.554407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d82d1c-4477-4d72-98c3-7fb897db2842 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:57.554908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d82d1c-4477-4d72-98c3-7fb897db2842 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:57.555422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d82d1c-4477-4d72-98c3-7fb897db2842 None admin] PATH_INFO: `/v3/services/network` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:57.556735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d82d1c-4477-4d72-98c3-7fb897db2842 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:57.559444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d82d1c-4477-4d72-98c3-7fb897db2842 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:57.577341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-d3d82d1c-4477-4d72-98c3-7fb897db2842 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 08:25:57.577341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:57.577341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:25:57.577341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:25:57.577341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:25:57.577341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:25:57.577341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:25:57.577341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:25:57.577341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 08:25:57.577341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:57.577341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:25:57.577341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:57.577341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:57.577341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:57.577341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:57.577341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:57.577341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:57.579579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:57.579579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:57.579579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:57.579579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:57.579579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:57.579579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:57.579579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:25:57.579579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:25:57.579579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:25:57.579579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:25:57.579579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:57.579579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:57.579579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:57.579579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:57.579579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:57.579579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:57.581469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:57.581469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:57.581469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:57.581469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:57.581469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:57.581469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:57.581469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:57.581469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:57.581469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:57.581469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:57.581469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:25:57.581469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:25:57.581469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 08:25:57.581469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:57.581469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 398/774] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:57 2022] GET /identity/v3/services/network => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:57.583481 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d82d1c-4477-4d72-98c3-7fb897db2842 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:57.642822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 377/775] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:25:57 2022] PUT /identity/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/84cba474fc7c470ea119de697b9d7ef9/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:25:57.644272 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aedf8e-2b40-424f-be51-cf12dcdb7fd7 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:57.871527 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aedf8e-2b40-424f-be51-cf12dcdb7fd7 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:57.898021 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d82d1c-4477-4d72-98c3-7fb897db2842 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:57.945270 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be43a20-9bad-4e4e-bb8f-d59f6982ed1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:57.946109 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be43a20-9bad-4e4e-bb8f-d59f6982ed1d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:57.946391 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be43a20-9bad-4e4e-bb8f-d59f6982ed1d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:57.946668 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be43a20-9bad-4e4e-bb8f-d59f6982ed1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:57.955269 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be43a20-9bad-4e4e-bb8f-d59f6982ed1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:57.956923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be43a20-9bad-4e4e-bb8f-d59f6982ed1d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:57.958525 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66b7ce-fa82-4ba1-b534-b810a0402e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:57.961327 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66b7ce-fa82-4ba1-b534-b810a0402e11 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:57.962127 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66b7ce-fa82-4ba1-b534-b810a0402e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:57.962847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66b7ce-fa82-4ba1-b534-b810a0402e11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:57.975644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-4be43a20-9bad-4e4e-bb8f-d59f6982ed1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:57.975644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:57.975644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:57.975644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:57.975644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:57.975644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:57.975644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:57.975644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:57.975644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:57.975644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:57.975644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:57.975644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:57.975644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:25:57.975644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:25:57.975644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:25:57.977451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:25:57.977451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:57.977451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:57.977451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:57.977451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:57.977451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:57.977451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:57.977451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:57.977451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:57.977451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:57.977451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:57.977451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:57.977451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:57.977451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:57.977451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:57.977451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:57.977451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:57.979250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:25:57.979250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:25:57.979250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:25:57.979250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:25:57.979250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:25:57.979250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:25:57.979250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:25:57.979250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:25:57.979250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 399/776] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:25:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:57.979250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be43a20-9bad-4e4e-bb8f-d59f6982ed1d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:57.985293 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66b7ce-fa82-4ba1-b534-b810a0402e11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:25:57.994653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-ce66b7ce-fa82-4ba1-b534-b810a0402e11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:25:57.994653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:57.994653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:57.994653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:57.994653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:57.994653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:57.994653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:57.994653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:57.994653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:57.994653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:57.994653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:57.994653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:57.994653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:25:57.994653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:25:57.994653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:25:57.994653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:25:57.997007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:25:57.997007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:25:57.997007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:25:57.997007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:25:57.997007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:57.997007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:57.997007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:57.997007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:57.997007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:57.997007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:57.997007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:57.997007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:57.997007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:57.997007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:57.997007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:57.997007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:57.998680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:57.998680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:57.998680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:57.998680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:57.998680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:25:57.998680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:25:57.998680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:25:57.998680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:25:57.998680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:25:57.998680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:25:57.998680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:25:57.998680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:58.000814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 378/777] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:25:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:58.000814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66b7ce-fa82-4ba1-b534-b810a0402e11 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:58.021017 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:58.043392 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:58.123769 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66b7ce-fa82-4ba1-b534-b810a0402e11 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:58.166761 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be43a20-9bad-4e4e-bb8f-d59f6982ed1d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:58.190099 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b55902-fad0-481e-9c2d-7fcfdc7c72d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:58.191422 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b55902-fad0-481e-9c2d-7fcfdc7c72d2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:58.191956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b55902-fad0-481e-9c2d-7fcfdc7c72d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:58.192442 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b55902-fad0-481e-9c2d-7fcfdc7c72d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:58.194574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b55902-fad0-481e-9c2d-7fcfdc7c72d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:58.195118 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b55902-fad0-481e-9c2d-7fcfdc7c72d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:58.198246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b55902-fad0-481e-9c2d-7fcfdc7c72d2 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:58.209510 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d5e9d-3118-4326-a03b-9a26a1362331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:58.210582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d5e9d-3118-4326-a03b-9a26a1362331 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:58.211067 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d5e9d-3118-4326-a03b-9a26a1362331 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:58.211567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d5e9d-3118-4326-a03b-9a26a1362331 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:58.212516 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d5e9d-3118-4326-a03b-9a26a1362331 None admin] RBAC: Adding query filter params (name=network) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:58.213090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d5e9d-3118-4326-a03b-9a26a1362331 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:58.214699 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d5e9d-3118-4326-a03b-9a26a1362331 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:58.220746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 400/778] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 18 08:25:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:58.221952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d5e9d-3118-4326-a03b-9a26a1362331 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:58.228319 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 379/779] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:25:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:58.230030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b55902-fad0-481e-9c2d-7fcfdc7c72d2 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:58.317389 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:58.358255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d5e9d-3118-4326-a03b-9a26a1362331 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:58.426019 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7090f26-a30a-4e97-9e28-1e643b46923f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:58.428261 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7090f26-a30a-4e97-9e28-1e643b46923f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:58.429057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7090f26-a30a-4e97-9e28-1e643b46923f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:58.430193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:58.430642 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7090f26-a30a-4e97-9e28-1e643b46923f None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:58.433920 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7090f26-a30a-4e97-9e28-1e643b46923f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:58.435438 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7090f26-a30a-4e97-9e28-1e643b46923f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:58.438498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7090f26-a30a-4e97-9e28-1e643b46923f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:58.443383 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b55902-fad0-481e-9c2d-7fcfdc7c72d2 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:58.461415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 401/780] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:25:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:58.463334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7090f26-a30a-4e97-9e28-1e643b46923f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:58.513955 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f749b6-d418-498e-aac0-8bc388a350c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:58.515029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f749b6-d418-498e-aac0-8bc388a350c2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:58.515537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f749b6-d418-498e-aac0-8bc388a350c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:58.516064 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f749b6-d418-498e-aac0-8bc388a350c2 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:58.517596 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f749b6-d418-498e-aac0-8bc388a350c2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:58.518142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f749b6-d418-498e-aac0-8bc388a350c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:58.521410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f749b6-d418-498e-aac0-8bc388a350c2 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:58.532486 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 380/781] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 18 08:25:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:58.534025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f749b6-d418-498e-aac0-8bc388a350c2 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:58.675381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7090f26-a30a-4e97-9e28-1e643b46923f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:58.742526 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f749b6-d418-498e-aac0-8bc388a350c2 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:58.751880 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81794a4e-3b11-42f4-af9c-ecbbe858d7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:58.753533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81794a4e-3b11-42f4-af9c-ecbbe858d7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:58.754305 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81794a4e-3b11-42f4-af9c-ecbbe858d7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:58.755124 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81794a4e-3b11-42f4-af9c-ecbbe858d7b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:58.757450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81794a4e-3b11-42f4-af9c-ecbbe858d7b8 None admin] RBAC: Adding query filter params (role.id=9fc4c3dcb90945ebb4eb5566e62e7833, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=2b03f8b8c51e472bbd4cf0afc9c6b632) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:58.758425 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81794a4e-3b11-42f4-af9c-ecbbe858d7b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:58.761623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81794a4e-3b11-42f4-af9c-ecbbe858d7b8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:58.774839 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 402/782] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:25:58 2022] GET /identity/v3/role_assignments?user.id=2b03f8b8c51e472bbd4cf0afc9c6b632&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=9fc4c3dcb90945ebb4eb5566e62e7833 => generated 259 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:58.776943 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81794a4e-3b11-42f4-af9c-ecbbe858d7b8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:58.783816 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:58.861099 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20d5ec-5f90-4d01-b15a-1dc37449c7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:58.863119 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20d5ec-5f90-4d01-b15a-1dc37449c7fe None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:58.864198 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20d5ec-5f90-4d01-b15a-1dc37449c7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:58.865031 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20d5ec-5f90-4d01-b15a-1dc37449c7fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:58.909123 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20d5ec-5f90-4d01-b15a-1dc37449c7fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:58.911812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20d5ec-5f90-4d01-b15a-1dc37449c7fe None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:58.975277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-1b20d5ec-5f90-4d01-b15a-1dc37449c7fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:58.975277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:58.975277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:58.975277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:58.975277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:58.975277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:58.975277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:58.975277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:58.975277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:58.975277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:58.975277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:58.975277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:58.975277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:25:58.975277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:25:58.975277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:25:58.977029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:25:58.977029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:58.977029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:58.977029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:58.977029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:58.977029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:58.977029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:58.977029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:58.977029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:58.977029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:58.977029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:58.977029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:58.977029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:58.977029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:58.977029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:58.977029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:58.977029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:58.978844 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:25:58.978844 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:25:58.978844 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:25:58.978844 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:58.980086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 381/783] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:25:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 447 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:58.989041 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20d5ec-5f90-4d01-b15a-1dc37449c7fe None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:59.002575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81794a4e-3b11-42f4-af9c-ecbbe858d7b8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:59.026501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:59.084488 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ece35-c809-487c-93db-d34ef3ffc112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:59.086113 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ece35-c809-487c-93db-d34ef3ffc112 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:59.086800 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ece35-c809-487c-93db-d34ef3ffc112 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:59.087486 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ece35-c809-487c-93db-d34ef3ffc112 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:59.089597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ece35-c809-487c-93db-d34ef3ffc112 None admin] RBAC: Adding query filter params (interface=public, service_id=b4ad8ede113c44ff9c5acdf1a7f9003e, region_id=RegionOne) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:59.090420 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ece35-c809-487c-93db-d34ef3ffc112 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:59.093501 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ece35-c809-487c-93db-d34ef3ffc112 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:59.105299 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 403/784] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 18 08:25:58 2022] GET /identity/v3/endpoints?service_id=b4ad8ede113c44ff9c5acdf1a7f9003e&interface=public®ion_id=RegionOne => generated 194 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:59.116884 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ece35-c809-487c-93db-d34ef3ffc112 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:59.335817 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20d5ec-5f90-4d01-b15a-1dc37449c7fe None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:59.374756 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:59.429150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dba34b-0335-497b-8a6c-3a2dc4575616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:59.430917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dba34b-0335-497b-8a6c-3a2dc4575616 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:59.431428 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dba34b-0335-497b-8a6c-3a2dc4575616 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:59.431973 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dba34b-0335-497b-8a6c-3a2dc4575616 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:59.433376 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dba34b-0335-497b-8a6c-3a2dc4575616 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:25:59.433982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dba34b-0335-497b-8a6c-3a2dc4575616 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:59.436964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dba34b-0335-497b-8a6c-3a2dc4575616 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:59.458920 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ece35-c809-487c-93db-d34ef3ffc112 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:59.462848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 382/785] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 474 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:59.465516 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec87da0-b5b1-4a1c-b11d-8b24ddd57a21 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:59.466027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec87da0-b5b1-4a1c-b11d-8b24ddd57a21 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:59.466531 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec87da0-b5b1-4a1c-b11d-8b24ddd57a21 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:59.469634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 383/786] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:25:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:25:59.471082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec87da0-b5b1-4a1c-b11d-8b24ddd57a21 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:59.526894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfe29a-dde9-4864-aa16-7799f2b66aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:59.534142 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfe29a-dde9-4864-aa16-7799f2b66aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:59.534142 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfe29a-dde9-4864-aa16-7799f2b66aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:59.534142 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfe29a-dde9-4864-aa16-7799f2b66aa3 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:59.534142 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dfe29a-dde9-4864-aa16-7799f2b66aa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:59.534142 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dfe29a-dde9-4864-aa16-7799f2b66aa3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:59.544241 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 404/787] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 18 08:25:59 2022] GET /identity/v3/services => generated 772 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:25:59.546449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4ff01-8687-4791-a3f0-a5a45c5827bf None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:59.547016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4ff01-8687-4791-a3f0-a5a45c5827bf None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:59.547610 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4ff01-8687-4791-a3f0-a5a45c5827bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:59.570745 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:59.579800 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-60c4ff01-8687-4791-a3f0-a5a45c5827bf None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:25:59.582143 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-60c4ff01-8687-4791-a3f0-a5a45c5827bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:25:59.623380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec87da0-b5b1-4a1c-b11d-8b24ddd57a21 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:25:59.763292 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72feb20-b6f8-4e60-9383-48ed396aac93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:25:59.764867 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72feb20-b6f8-4e60-9383-48ed396aac93 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:25:59.765613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72feb20-b6f8-4e60-9383-48ed396aac93 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:25:59.765936 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72feb20-b6f8-4e60-9383-48ed396aac93 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:25:59.789116 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72feb20-b6f8-4e60-9383-48ed396aac93 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:25:59.792224 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72feb20-b6f8-4e60-9383-48ed396aac93 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:25:59.793626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 405/788] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:25:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:25:59.818700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4ff01-8687-4791-a3f0-a5a45c5827bf None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:59.820201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-e72feb20-b6f8-4e60-9383-48ed396aac93 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 08:25:59.820201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:25:59.820201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:25:59.820201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:25:59.820201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:25:59.820201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:25:59.820201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:25:59.820201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:25:59.820201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:25:59.820201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:25:59.820201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:25:59.820201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:25:59.820201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:25:59.820201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:25:59.820201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:25:59.820201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:25:59.827689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:25:59.827689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:25:59.827689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:25:59.827689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:59.827689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:25:59.827689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:25:59.827689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:25:59.827689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:25:59.827689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:25:59.827689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:25:59.827689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:25:59.827689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:25:59.827689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:25:59.827689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:25:59.827689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:25:59.827689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:25:59.827689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:25:59.827689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:25:59.828880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:25:59.828880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:25:59.828880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:25:59.828880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:25:59.828880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:25:59.828880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:25:59.828880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:25:59.828880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:25:59.828880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 08:25:59.828880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:25:59.828880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 384/789] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:25:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:25:59.836487 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72feb20-b6f8-4e60-9383-48ed396aac93 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:25:59.940825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:25:59.984717 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4ff01-8687-4791-a3f0-a5a45c5827bf None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:00.025062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72feb20-b6f8-4e60-9383-48ed396aac93 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:00.032679 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db348ae9-5a50-484f-9f4b-e78614118036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:00.033883 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db348ae9-5a50-484f-9f4b-e78614118036 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:00.034376 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db348ae9-5a50-484f-9f4b-e78614118036 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:00.034856 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db348ae9-5a50-484f-9f4b-e78614118036 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:00.035793 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db348ae9-5a50-484f-9f4b-e78614118036 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:00.037490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db348ae9-5a50-484f-9f4b-e78614118036 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:00.050939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-db348ae9-5a50-484f-9f4b-e78614118036 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 08:26:00.050939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:00.050939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:26:00.050939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:26:00.050939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:26:00.050939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:26:00.050939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:26:00.050939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:00.050939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 08:26:00.050939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:00.050939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:00.050939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:00.050939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:00.050939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:00.050939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:00.050939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:00.052537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:00.052537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:00.052537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:00.052537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:00.052537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:00.052537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:00.052537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:00.052537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:26:00.052537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:26:00.052537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:26:00.052537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:26:00.052537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:00.052537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:00.052537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:00.052537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:00.052537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:00.053575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:00.053575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:00.053575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:00.053575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:00.053575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:00.053575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:00.053575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:00.053575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:00.053575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:00.053575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:00.053575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:00.053575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:26:00.053575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:00.053575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 08:26:00.053575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:00.056316 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 406/790] 10.222.0.57 () {56 vars in 1129 bytes} [Mon Jul 18 08:25:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:00.066432 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db348ae9-5a50-484f-9f4b-e78614118036 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:00.078883 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:00.090195 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28ecf9-4411-4d3b-b8ce-a93f8e405cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:00.091451 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28ecf9-4411-4d3b-b8ce-a93f8e405cdb None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:00.092060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28ecf9-4411-4d3b-b8ce-a93f8e405cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:00.092658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28ecf9-4411-4d3b-b8ce-a93f8e405cdb None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:00.093682 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28ecf9-4411-4d3b-b8ce-a93f8e405cdb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:00.094174 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28ecf9-4411-4d3b-b8ce-a93f8e405cdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:00.095997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28ecf9-4411-4d3b-b8ce-a93f8e405cdb None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:00.121033 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 385/791] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Jul 18 08:25:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:00.123340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6f647-40a8-4bd0-b88c-e7948c26a807 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:00.123651 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6f647-40a8-4bd0-b88c-e7948c26a807 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:00.123972 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6f647-40a8-4bd0-b88c-e7948c26a807 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:00.125686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 386/792] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:00.130755 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6f647-40a8-4bd0-b88c-e7948c26a807 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:00.186803 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db348ae9-5a50-484f-9f4b-e78614118036 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:00.216823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:00.227139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18251624-2292-4be5-be38-67f73afddff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:00.227427 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18251624-2292-4be5-be38-67f73afddff3 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:00.227700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18251624-2292-4be5-be38-67f73afddff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:00.229855 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18251624-2292-4be5-be38-67f73afddff3 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:00.229855 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18251624-2292-4be5-be38-67f73afddff3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:00.229855 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18251624-2292-4be5-be38-67f73afddff3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:00.230457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18251624-2292-4be5-be38-67f73afddff3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:00.236325 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 407/793] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Jul 18 08:26:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:00.237815 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cf5df-15ce-46bf-8814-3284a7e78650 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:00.238112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cf5df-15ce-46bf-8814-3284a7e78650 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:00.238651 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048cf5df-15ce-46bf-8814-3284a7e78650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:00.256662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-048cf5df-15ce-46bf-8814-3284a7e78650 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:00.257930 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6f647-40a8-4bd0-b88c-e7948c26a807 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:00.258048 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-048cf5df-15ce-46bf-8814-3284a7e78650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:00.299650 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad083bfd-0af8-446b-915e-c56573826a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:00.300559 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad083bfd-0af8-446b-915e-c56573826a38 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:00.300841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad083bfd-0af8-446b-915e-c56573826a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:00.301115 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad083bfd-0af8-446b-915e-c56573826a38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:00.313734 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad083bfd-0af8-446b-915e-c56573826a38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:00.315336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad083bfd-0af8-446b-915e-c56573826a38 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:00.332504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 408/794] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:00.334192 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048cf5df-15ce-46bf-8814-3284a7e78650 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:00.336232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-ad083bfd-0af8-446b-915e-c56573826a38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:00.336232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:00.336232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:00.336232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:00.336232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:00.336232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:00.336232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:00.336232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:00.336232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:00.336232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:00.336232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:00.336232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:00.336232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:00.336232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:00.336232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:00.337133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:00.337133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:00.337133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:00.337133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:00.337133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:00.337133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:00.337133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:00.337133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:00.337133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:00.337133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:00.337133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:00.337133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:00.337133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:00.337133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:00.337133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:00.337133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:00.337133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:00.338039 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:00.338039 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:00.338039 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:00.338039 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:00.342425 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 387/795] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:00.343715 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad083bfd-0af8-446b-915e-c56573826a38 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:00.398240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:00.453442 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048cf5df-15ce-46bf-8814-3284a7e78650 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:00.470365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad083bfd-0af8-446b-915e-c56573826a38 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:00.479462 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:00.496031 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417fefd-9ee4-48d0-b77f-e0515fa95c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:00.496726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417fefd-9ee4-48d0-b77f-e0515fa95c37 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:00.497015 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417fefd-9ee4-48d0-b77f-e0515fa95c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:00.497294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417fefd-9ee4-48d0-b77f-e0515fa95c37 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:00.498029 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417fefd-9ee4-48d0-b77f-e0515fa95c37 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:00.498343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417fefd-9ee4-48d0-b77f-e0515fa95c37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:00.500187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417fefd-9ee4-48d0-b77f-e0515fa95c37 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:00.506010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 409/796] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Jul 18 08:26:00 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:00.506988 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417fefd-9ee4-48d0-b77f-e0515fa95c37 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:00.517454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cbccd-c142-4a79-8113-c6fe1a2b511d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:00.518501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cbccd-c142-4a79-8113-c6fe1a2b511d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:00.518974 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cbccd-c142-4a79-8113-c6fe1a2b511d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:00.519437 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cbccd-c142-4a79-8113-c6fe1a2b511d None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:00.520439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cbccd-c142-4a79-8113-c6fe1a2b511d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:00.521203 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cbccd-c142-4a79-8113-c6fe1a2b511d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:00.522833 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cbccd-c142-4a79-8113-c6fe1a2b511d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:00.541884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 388/797] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:00.543168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cbccd-c142-4a79-8113-c6fe1a2b511d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:00.629720 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417fefd-9ee4-48d0-b77f-e0515fa95c37 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:00.658367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:00.663013 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cbccd-c142-4a79-8113-c6fe1a2b511d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:00.685688 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b597826-a951-40cd-9c77-aa16ed20364c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:00.687201 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b597826-a951-40cd-9c77-aa16ed20364c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:00.687736 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b597826-a951-40cd-9c77-aa16ed20364c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:00.688116 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b597826-a951-40cd-9c77-aa16ed20364c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:00.704029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbec96d-f677-4785-b51f-cad1fbcb8bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:00.705020 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbec96d-f677-4785-b51f-cad1fbcb8bf5 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:00.705484 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbec96d-f677-4785-b51f-cad1fbcb8bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:00.705937 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbec96d-f677-4785-b51f-cad1fbcb8bf5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:00.706910 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbec96d-f677-4785-b51f-cad1fbcb8bf5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:00.708132 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b597826-a951-40cd-9c77-aa16ed20364c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:26:00.709040 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbec96d-f677-4785-b51f-cad1fbcb8bf5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:00.717915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-7b597826-a951-40cd-9c77-aa16ed20364c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:26:00.717915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:00.717915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:00.717915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:00.717915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:00.717915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:00.717915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:00.717915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:00.717915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:00.717915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:00.717915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:00.717915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:00.717915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:26:00.717915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:26:00.717915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:26:00.717915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:26:00.718923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:26:00.718923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:26:00.718923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:26:00.718923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:26:00.718923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:00.718923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:00.718923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:00.718923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:00.718923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:00.718923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:00.718923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:00.718923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:00.718923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:00.718923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:00.718923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:00.718923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:00.719799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:00.719799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:00.719799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:00.719799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:00.719799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:26:00.719799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:26:00.719799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:26:00.719799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:26:00.719799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:26:00.719799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:26:00.719799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:26:00.719799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:00.720508 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 410/798] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:26:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:00.721119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b597826-a951-40cd-9c77-aa16ed20364c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:00.777017 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:00.857128 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 389/799] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 18 08:26:00 2022] POST /identity/v3/endpoints => generated 346 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:26:00.857128 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbec96d-f677-4785-b51f-cad1fbcb8bf5 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:00.873747 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b597826-a951-40cd-9c77-aa16ed20364c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:00.976064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4ba53-edda-458b-9700-376e3eada073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:00.977163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4ba53-edda-458b-9700-376e3eada073 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:00.977606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4ba53-edda-458b-9700-376e3eada073 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:00.978034 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4ba53-edda-458b-9700-376e3eada073 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:00.999306 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4ba53-edda-458b-9700-376e3eada073 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:01.001154 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4ba53-edda-458b-9700-376e3eada073 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:01.030886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-6ac4ba53-edda-458b-9700-376e3eada073 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:01.030886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:01.030886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:01.030886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:01.030886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:01.030886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:01.030886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:01.030886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:01.030886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:01.030886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:01.030886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:01.030886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:01.030886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:26:01.030886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:26:01.030886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:26:01.032024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:26:01.032024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:01.032024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:01.032024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:01.032024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:01.032024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:01.032024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:01.032024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:01.032024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:01.032024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:01.032024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:01.032024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:01.032024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:01.032024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:01.032024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:01.032024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:01.032024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:01.033212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:26:01.033212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:26:01.033212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:26:01.033212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:26:01.033212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:26:01.033212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:26:01.033212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:01.033212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:01.035594 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 411/800] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:26:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:01.045660 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4ba53-edda-458b-9700-376e3eada073 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:01.121664 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbec96d-f677-4785-b51f-cad1fbcb8bf5 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:01.182961 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:01.186099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4ba53-edda-458b-9700-376e3eada073 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:01.196841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bfe45-56bc-420d-9b34-cca1e33753b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:01.198278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bfe45-56bc-420d-9b34-cca1e33753b7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:01.198989 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bfe45-56bc-420d-9b34-cca1e33753b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:01.199759 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bfe45-56bc-420d-9b34-cca1e33753b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:01.208311 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383bfe45-56bc-420d-9b34-cca1e33753b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:01.208311 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383bfe45-56bc-420d-9b34-cca1e33753b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:01.208311 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383bfe45-56bc-420d-9b34-cca1e33753b7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:01.221103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 390/801] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:26:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:01.228142 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fbee8-8eba-4c75-8ed2-42dce2d84fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:01.228731 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fbee8-8eba-4c75-8ed2-42dce2d84fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:01.229020 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fbee8-8eba-4c75-8ed2-42dce2d84fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:01.229303 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fbee8-8eba-4c75-8ed2-42dce2d84fa5 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:01.230115 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fbee8-8eba-4c75-8ed2-42dce2d84fa5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:01.230432 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fbee8-8eba-4c75-8ed2-42dce2d84fa5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:01.232018 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fbee8-8eba-4c75-8ed2-42dce2d84fa5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:01.234976 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bfe45-56bc-420d-9b34-cca1e33753b7 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:01.248028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 412/802] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:26:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:01.259352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fbee8-8eba-4c75-8ed2-42dce2d84fa5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:01.279491 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:01.377524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fbee8-8eba-4c75-8ed2-42dce2d84fa5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:01.417719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca5272-c1e6-4d56-8b51-07aaa2606be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:01.418854 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca5272-c1e6-4d56-8b51-07aaa2606be5 None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:01.419284 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca5272-c1e6-4d56-8b51-07aaa2606be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:01.419720 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca5272-c1e6-4d56-8b51-07aaa2606be5 None admin] PATH_INFO: `/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/09a5c2e804504038817cad76544dce5f/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:01.422036 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca5272-c1e6-4d56-8b51-07aaa2606be5 None admin] RBAC: Authorizing `identity:create_grant(project_id=35c846931fd443fbb83a6fbe0aeaa866, user_id=09a5c2e804504038817cad76544dce5f, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:01.423623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca5272-c1e6-4d56-8b51-07aaa2606be5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:01.451380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bfe45-56bc-420d-9b34-cca1e33753b7 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:01.495014 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b3f1a-55ed-4443-b54a-097539ed8dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:01.495762 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b3f1a-55ed-4443-b54a-097539ed8dbb None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:01.496049 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b3f1a-55ed-4443-b54a-097539ed8dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:01.496329 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b3f1a-55ed-4443-b54a-097539ed8dbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:01.508934 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 413/803] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:26:01 2022] PUT /identity/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/09a5c2e804504038817cad76544dce5f/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:26:01.527350 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b3f1a-55ed-4443-b54a-097539ed8dbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:01.530831 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b3f1a-55ed-4443-b54a-097539ed8dbb None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:01.562013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79532a-fade-4a32-ba17-7b6c8505d955 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:01.563073 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79532a-fade-4a32-ba17-7b6c8505d955 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:01.563073 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79532a-fade-4a32-ba17-7b6c8505d955 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:01.564328 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 414/804] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:26:01.570144 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1b779-9e08-4bd3-8d9e-42ddb15ccaff None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:01.570434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1b779-9e08-4bd3-8d9e-42ddb15ccaff None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:01.570804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1b779-9e08-4bd3-8d9e-42ddb15ccaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:01.578528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-d97b3f1a-55ed-4443-b54a-097539ed8dbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:01.578528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:01.578528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:01.578528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:01.578528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:01.578528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:01.578528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:01.578528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:01.578528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:01.578528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:01.578528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:01.578528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:01.578528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:01.578528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:01.578528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:01.580395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:01.580395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:01.580395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:01.580395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:01.580395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:01.580395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:01.580395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:01.580395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:01.580395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:01.580395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:01.580395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:01.580395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:01.580395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:01.580395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:01.580395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:01.580395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:01.580395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:01.582217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:01.582217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:01.582217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:01.582217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:01.587133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 391/805] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:01.597396 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b3f1a-55ed-4443-b54a-097539ed8dbb None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:01.635954 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ced1b779-9e08-4bd3-8d9e-42ddb15ccaff None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:01.637237 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ced1b779-9e08-4bd3-8d9e-42ddb15ccaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:01.685429 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:01.754814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:01.774772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 415/806] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:01.798797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1b779-9e08-4bd3-8d9e-42ddb15ccaff None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:01.818983 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b3f1a-55ed-4443-b54a-097539ed8dbb None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:01.860134 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c62f4c-9231-4c27-b758-cb88dbe30c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:01.861136 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c62f4c-9231-4c27-b758-cb88dbe30c5d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:01.861136 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c62f4c-9231-4c27-b758-cb88dbe30c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:01.861384 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c62f4c-9231-4c27-b758-cb88dbe30c5d None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:01.862093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c62f4c-9231-4c27-b758-cb88dbe30c5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:01.862401 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c62f4c-9231-4c27-b758-cb88dbe30c5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:01.864127 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c62f4c-9231-4c27-b758-cb88dbe30c5d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:01.877885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 392/807] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:01.890596 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c62f4c-9231-4c27-b758-cb88dbe30c5d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:02.018104 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1b779-9e08-4bd3-8d9e-42ddb15ccaff None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:02.026066 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c62f4c-9231-4c27-b758-cb88dbe30c5d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:02.085167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:02.097499 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fc9bb-7b88-48b4-8f01-166f6b986d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:02.099982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fc9bb-7b88-48b4-8f01-166f6b986d1a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:02.100821 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fc9bb-7b88-48b4-8f01-166f6b986d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:02.101517 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fc9bb-7b88-48b4-8f01-166f6b986d1a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:02.118376 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633fc9bb-7b88-48b4-8f01-166f6b986d1a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:02.121490 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633fc9bb-7b88-48b4-8f01-166f6b986d1a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:02.122577 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9887a8-722e-42cf-b740-eecc102ee4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:02.123768 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9887a8-722e-42cf-b740-eecc102ee4d7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:02.124169 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9887a8-722e-42cf-b740-eecc102ee4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:02.124573 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9887a8-722e-42cf-b740-eecc102ee4d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:02.160352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-633fc9bb-7b88-48b4-8f01-166f6b986d1a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 08:26:02.160352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:02.160352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:02.160352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:02.160352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:02.160352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:02.160352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:02.160352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:02.160352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:02.160352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:02.160352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:02.160352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:02.160352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:26:02.160352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:26:02.160352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:26:02.160352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:26:02.162814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:02.162814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:02.162814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:26:02.162814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:02.162814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:26:02.162814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:02.162814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:02.162814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:02.162814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:02.162814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:02.162814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:02.162814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:02.162814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:02.162814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:02.162814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:02.162814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:02.162814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:02.162814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:02.164673 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:02.164673 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:02.164673 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:26:02.164673 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:26:02.164673 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:26:02.164673 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:26:02.164673 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:26:02.164673 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:26:02.164673 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 08:26:02.164673 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:02.167003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9887a8-722e-42cf-b740-eecc102ee4d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:26:02.169270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 393/808] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 18 08:26:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:02.178377 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fc9bb-7b88-48b4-8f01-166f6b986d1a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:02.178623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-af9887a8-722e-42cf-b740-eecc102ee4d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:26:02.178623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:02.178623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:02.178623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:02.178623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:02.178623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:02.178623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:02.178623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:02.178623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:02.178623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:02.178623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:02.178623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:02.178623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:26:02.178623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:26:02.178623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:26:02.178623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:26:02.179519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:26:02.179519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:26:02.179519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:26:02.179519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:26:02.179519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:02.179519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:02.179519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:02.179519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:02.179519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:02.179519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:02.179519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:02.179519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:02.179519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:02.179519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:02.179519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:02.179519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:02.180409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:02.180409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:02.180409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:02.180409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:02.180409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:26:02.180409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:26:02.180409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:26:02.180409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:26:02.180409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:26:02.180409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:26:02.180409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:26:02.180409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:02.184135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 416/809] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:26:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:02.199333 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9887a8-722e-42cf-b740-eecc102ee4d7 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:02.230048 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:02.376537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fc9bb-7b88-48b4-8f01-166f6b986d1a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:02.414606 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a06e6-19e9-408d-97ce-39df1a737776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:02.415293 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a06e6-19e9-408d-97ce-39df1a737776 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:02.415571 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a06e6-19e9-408d-97ce-39df1a737776 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:02.415894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a06e6-19e9-408d-97ce-39df1a737776 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:02.416748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a06e6-19e9-408d-97ce-39df1a737776 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:02.417053 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a06e6-19e9-408d-97ce-39df1a737776 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:02.418092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9887a8-722e-42cf-b740-eecc102ee4d7 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:02.418526 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a06e6-19e9-408d-97ce-39df1a737776 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:02.439546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 394/810] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 18 08:26:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:02.441068 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d686e-7f90-466a-b520-84208394da73 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:02.441362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d686e-7f90-466a-b520-84208394da73 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:02.441641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d686e-7f90-466a-b520-84208394da73 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:02.444302 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 395/811] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:02.449304 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399e355-b5ae-400d-b343-6844caebf1d0 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:02.449727 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399e355-b5ae-400d-b343-6844caebf1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:02.450641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399e355-b5ae-400d-b343-6844caebf1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:02.472041 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a399e355-b5ae-400d-b343-6844caebf1d0 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:02.474363 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a399e355-b5ae-400d-b343-6844caebf1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:02.492611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56dd38-2935-448f-ba7c-4944b13a65a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:02.494468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56dd38-2935-448f-ba7c-4944b13a65a4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:02.495338 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56dd38-2935-448f-ba7c-4944b13a65a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:02.496084 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56dd38-2935-448f-ba7c-4944b13a65a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:02.497737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af56dd38-2935-448f-ba7c-4944b13a65a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:02.498602 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af56dd38-2935-448f-ba7c-4944b13a65a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:02.501692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af56dd38-2935-448f-ba7c-4944b13a65a4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:02.523899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 417/812] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:26:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:02.535564 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56dd38-2935-448f-ba7c-4944b13a65a4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:02.535989 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:02.566587 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 396/813] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:02.568881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399e355-b5ae-400d-b343-6844caebf1d0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:02.677863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:02.714425 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399e355-b5ae-400d-b343-6844caebf1d0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:02.731379 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56dd38-2935-448f-ba7c-4944b13a65a4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:02.769602 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ae731-98e4-40c9-a334-ce0fa0554ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:02.770417 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ae731-98e4-40c9-a334-ce0fa0554ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:02.771389 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ae731-98e4-40c9-a334-ce0fa0554ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:02.771697 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ae731-98e4-40c9-a334-ce0fa0554ea9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:02.787417 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ae731-98e4-40c9-a334-ce0fa0554ea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:02.789250 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ae731-98e4-40c9-a334-ce0fa0554ea9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:02.800386 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c685f3-e931-49f3-98a7-9a1295c8e9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:02.802074 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c685f3-e931-49f3-98a7-9a1295c8e9db None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:02.802878 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c685f3-e931-49f3-98a7-9a1295c8e9db None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:02.803661 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c685f3-e931-49f3-98a7-9a1295c8e9db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:02.812109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-704ae731-98e4-40c9-a334-ce0fa0554ea9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:02.812109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:02.812109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:02.812109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:02.812109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:02.812109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:02.812109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:02.812109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:02.812109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:02.812109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:02.812109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:02.812109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:02.812109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:02.812109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:02.812109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:02.813861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:02.813861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:02.813861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:02.813861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:02.813861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:02.813861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:02.813861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:02.813861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:02.813861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:02.813861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:02.813861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:02.813861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:02.813861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:02.813861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:02.813861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:02.813861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:02.813861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:02.815707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:02.815707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:02.815707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:02.815707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:02.819395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 397/814] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:02.820249 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ae731-98e4-40c9-a334-ce0fa0554ea9 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:02.861049 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:02.954655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c685f3-e931-49f3-98a7-9a1295c8e9db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:02.958657 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c685f3-e931-49f3-98a7-9a1295c8e9db None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:02.961186 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ae731-98e4-40c9-a334-ce0fa0554ea9 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:03.004379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f47b5-a605-4f95-908f-625dd6ca5a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:03.005729 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f47b5-a605-4f95-908f-625dd6ca5a23 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:03.006009 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f47b5-a605-4f95-908f-625dd6ca5a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:03.006278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f47b5-a605-4f95-908f-625dd6ca5a23 None admin] PATH_INFO: `/v3/services/network` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:03.006962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f47b5-a605-4f95-908f-625dd6ca5a23 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:03.008661 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f47b5-a605-4f95-908f-625dd6ca5a23 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:03.025907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-f30f47b5-a605-4f95-908f-625dd6ca5a23 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 08:26:03.025907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:03.025907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:26:03.025907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:26:03.025907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:26:03.025907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:26:03.025907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:26:03.025907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:03.025907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 08:26:03.025907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:03.025907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:03.025907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:03.025907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:03.025907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:03.025907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:03.025907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:03.025907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:03.028010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:03.028010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:03.028010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:03.028010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:03.028010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:03.028010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:03.028010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:26:03.028010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:26:03.028010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:26:03.028010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:26:03.028010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:03.028010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:03.028010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:03.028010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:03.028010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:03.028010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:03.029718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:03.029718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:03.029718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:03.029718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:03.029718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:03.029718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:03.029718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:03.029718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:03.029718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:03.029718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:03.029718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:26:03.029718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:03.029718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 08:26:03.029718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:03.029718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 398/815] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:26:02 2022] GET /identity/v3/services/network => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:03.032681 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f47b5-a605-4f95-908f-625dd6ca5a23 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:03.113631 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:03.151198 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:03.159073 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f47b5-a605-4f95-908f-625dd6ca5a23 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:03.180717 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 418/816] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:26:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 08:26:03.182752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c685f3-e931-49f3-98a7-9a1295c8e9db None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:03.204153 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc03133-fede-4abf-97a2-942230806821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:03.204902 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc03133-fede-4abf-97a2-942230806821 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:03.205159 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc03133-fede-4abf-97a2-942230806821 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:03.205397 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc03133-fede-4abf-97a2-942230806821 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:03.206122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc03133-fede-4abf-97a2-942230806821 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:03.206432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc03133-fede-4abf-97a2-942230806821 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:03.209116 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc03133-fede-4abf-97a2-942230806821 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:03.227825 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 399/817] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:03.227825 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc03133-fede-4abf-97a2-942230806821 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:03.351244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc03133-fede-4abf-97a2-942230806821 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:03.394601 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67b174-e1fc-461f-9c71-47082dea47b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:03.395591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67b174-e1fc-461f-9c71-47082dea47b5 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:03.395938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67b174-e1fc-461f-9c71-47082dea47b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:03.396340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67b174-e1fc-461f-9c71-47082dea47b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:03.404768 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c685f3-e931-49f3-98a7-9a1295c8e9db None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:03.409348 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67b174-e1fc-461f-9c71-47082dea47b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:03.412496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67b174-e1fc-461f-9c71-47082dea47b5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:03.433591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-bc67b174-e1fc-461f-9c71-47082dea47b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:03.433591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:03.433591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:03.433591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:03.433591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:03.433591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:03.433591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:03.433591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:03.433591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:03.433591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:03.433591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:03.433591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:03.433591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:03.433591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:03.433591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:03.435400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:03.435400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:03.435400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:03.435400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:03.435400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:03.435400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:03.435400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:03.435400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:03.435400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:03.435400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:03.435400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:03.435400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:03.435400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:03.435400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:03.435400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:03.435400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:03.435400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:03.437494 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:03.437494 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:03.437494 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:03.437494 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:03.437494 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 400/818] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:03.437494 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67b174-e1fc-461f-9c71-47082dea47b5 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:03.449781 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:03.483207 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b988d-57de-4799-9748-956508c0fb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:03.484018 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b988d-57de-4799-9748-956508c0fb46 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:03.484412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b988d-57de-4799-9748-956508c0fb46 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:03.484803 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b988d-57de-4799-9748-956508c0fb46 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:03.485982 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b988d-57de-4799-9748-956508c0fb46 None admin] RBAC: Adding query filter params (name=network) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:03.486404 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b988d-57de-4799-9748-956508c0fb46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:03.488462 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b988d-57de-4799-9748-956508c0fb46 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:03.496238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 419/819] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 18 08:26:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:03.497603 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b988d-57de-4799-9748-956508c0fb46 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:03.542739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:03.632669 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67b174-e1fc-461f-9c71-47082dea47b5 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:03.665457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b988d-57de-4799-9748-956508c0fb46 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:03.671788 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dc3e5-a57f-49e0-81c9-842b6b07f885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:03.672643 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dc3e5-a57f-49e0-81c9-842b6b07f885 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:03.672933 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dc3e5-a57f-49e0-81c9-842b6b07f885 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:03.673212 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dc3e5-a57f-49e0-81c9-842b6b07f885 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:03.683028 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dc3e5-a57f-49e0-81c9-842b6b07f885 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:03.684552 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dc3e5-a57f-49e0-81c9-842b6b07f885 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:03.701582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-b10dc3e5-a57f-49e0-81c9-842b6b07f885 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 08:26:03.701582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:03.701582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:03.701582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:03.701582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:03.701582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:03.701582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:03.701582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:03.701582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:03.701582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:03.701582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:03.701582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:03.701582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:26:03.701582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:26:03.701582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:26:03.701582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:26:03.702694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:03.702694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:03.702694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:26:03.702694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:03.702694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:26:03.702694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:03.702694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:03.702694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:03.702694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:03.702694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:03.702694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:03.702694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:03.702694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:03.702694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:03.702694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:03.702694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:03.702694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:03.702694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:03.703657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:03.703657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:03.703657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:26:03.703657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:26:03.703657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:26:03.703657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:26:03.703657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:26:03.703657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:26:03.703657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 08:26:03.703657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:03.703657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 401/820] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 18 08:26:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:03.703657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dc3e5-a57f-49e0-81c9-842b6b07f885 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:03.735713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3412e-8452-4071-851d-1864421af741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:03.736348 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3412e-8452-4071-851d-1864421af741 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:03.737334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3412e-8452-4071-851d-1864421af741 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:03.737623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3412e-8452-4071-851d-1864421af741 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:03.738636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3412e-8452-4071-851d-1864421af741 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:03.738953 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3412e-8452-4071-851d-1864421af741 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:03.741559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3412e-8452-4071-851d-1864421af741 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:03.759157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 420/821] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:03.759157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb43d0d-a3b8-4f2e-8f33-752422b518af None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:03.759157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb43d0d-a3b8-4f2e-8f33-752422b518af None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:03.759157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb43d0d-a3b8-4f2e-8f33-752422b518af None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:03.759157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 421/822] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:03.759626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb43d0d-a3b8-4f2e-8f33-752422b518af None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:03.807154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:03.848458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dc3e5-a57f-49e0-81c9-842b6b07f885 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:03.892116 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:03.899011 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb43d0d-a3b8-4f2e-8f33-752422b518af None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:03.900719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7fb2e-e4a5-41f0-888c-4962b743713f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:03.902021 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7fb2e-e4a5-41f0-888c-4962b743713f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:03.902516 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7fb2e-e4a5-41f0-888c-4962b743713f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:03.903006 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7fb2e-e4a5-41f0-888c-4962b743713f None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:03.904376 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7fb2e-e4a5-41f0-888c-4962b743713f None admin] RBAC: Adding query filter params (type=network) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:03.904919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7fb2e-e4a5-41f0-888c-4962b743713f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:03.907563 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7fb2e-e4a5-41f0-888c-4962b743713f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:03.916895 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 402/823] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 18 08:26:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:03.918291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7fb2e-e4a5-41f0-888c-4962b743713f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:03.937143 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f7868-a6a2-4f4a-9c96-748634014442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:03.937823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f7868-a6a2-4f4a-9c96-748634014442 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:03.938107 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f7868-a6a2-4f4a-9c96-748634014442 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:03.938381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f7868-a6a2-4f4a-9c96-748634014442 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:03.939238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f7868-a6a2-4f4a-9c96-748634014442 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:03.939545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f7868-a6a2-4f4a-9c96-748634014442 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:03.941033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f7868-a6a2-4f4a-9c96-748634014442 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:03.961361 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 422/824] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Jul 18 08:26:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:03.962753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13288de6-b9c5-4e6f-a438-15cdcc72c032 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:03.963046 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13288de6-b9c5-4e6f-a438-15cdcc72c032 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:03.963319 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13288de6-b9c5-4e6f-a438-15cdcc72c032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:04.081880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7fb2e-e4a5-41f0-888c-4962b743713f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:04.101207 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-13288de6-b9c5-4e6f-a438-15cdcc72c032 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:04.102433 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-13288de6-b9c5-4e6f-a438-15cdcc72c032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:04.114708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:04.121441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9c531-3eeb-4b78-94fc-31e493573a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:04.132730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9c531-3eeb-4b78-94fc-31e493573a04 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:04.132730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9c531-3eeb-4b78-94fc-31e493573a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:04.132730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9c531-3eeb-4b78-94fc-31e493573a04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:04.132730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9c531-3eeb-4b78-94fc-31e493573a04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:04.133452 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9c531-3eeb-4b78-94fc-31e493573a04 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:04.149391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-3be9c531-3eeb-4b78-94fc-31e493573a04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:04.149391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:04.149391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:04.149391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:04.149391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:04.149391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:04.149391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:04.149391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:04.149391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:04.149391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:04.149391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:04.149391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:04.149391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:26:04.149391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:26:04.149391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:26:04.151379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:26:04.151379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:04.151379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:04.151379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:04.151379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:04.151379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:04.151379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:04.151379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:04.151379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:04.151379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:04.151379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:04.151379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:04.151379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:04.151379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:04.151379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:04.151379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:04.151379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:04.153226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:26:04.153226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:26:04.153226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:26:04.153226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:26:04.153226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:26:04.153226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:26:04.153226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:04.153226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:04.155694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 403/825] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:26:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:04.156948 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7fcce-2a7f-4ba3-83a5-7d747eff57fb None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:04.157238 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7fcce-2a7f-4ba3-83a5-7d747eff57fb None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:04.157514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7fcce-2a7f-4ba3-83a5-7d747eff57fb None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:04.158874 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 404/826] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:04.159560 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7fcce-2a7f-4ba3-83a5-7d747eff57fb None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:04.192732 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 423/827] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:04.194028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13288de6-b9c5-4e6f-a438-15cdcc72c032 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:04.299091 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:04.318151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13288de6-b9c5-4e6f-a438-15cdcc72c032 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:04.319054 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7fcce-2a7f-4ba3-83a5-7d747eff57fb None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:04.366808 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06754786-b78d-45fb-a5dc-19eb99437892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:04.367085 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79582b0-2e0b-4ff0-b84b-08a4c15e01da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:04.367770 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06754786-b78d-45fb-a5dc-19eb99437892 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:04.367995 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79582b0-2e0b-4ff0-b84b-08a4c15e01da None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:04.368234 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06754786-b78d-45fb-a5dc-19eb99437892 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:04.368456 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79582b0-2e0b-4ff0-b84b-08a4c15e01da None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:04.368684 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06754786-b78d-45fb-a5dc-19eb99437892 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:04.368902 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79582b0-2e0b-4ff0-b84b-08a4c15e01da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:04.369992 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06754786-b78d-45fb-a5dc-19eb99437892 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:04.372381 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06754786-b78d-45fb-a5dc-19eb99437892 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:04.385687 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79582b0-2e0b-4ff0-b84b-08a4c15e01da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:04.387246 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79582b0-2e0b-4ff0-b84b-08a4c15e01da None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:04.406689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f79582b0-2e0b-4ff0-b84b-08a4c15e01da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:04.406689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:04.406689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:04.406689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:04.406689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:04.406689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:04.406689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:04.406689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:04.406689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:04.406689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:04.406689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:04.406689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:04.406689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:04.406689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:04.406689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:04.408473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:04.408473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:04.408473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:04.408473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:04.408473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:04.408473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:04.408473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:04.408473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:04.408473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:04.408473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:04.408473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:04.408473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:04.408473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:04.408473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:04.408473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:04.408473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:04.408473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:04.410172 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:04.410172 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:04.410172 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:04.410172 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:04.410172 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 424/828] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:04.410172 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79582b0-2e0b-4ff0-b84b-08a4c15e01da None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:04.454499 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 405/829] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 18 08:26:04 2022] POST /identity/v3/endpoints => generated 327 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:26:04.456702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b24bec-196d-43fb-8459-49f7d26d51af None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:04.457185 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b24bec-196d-43fb-8459-49f7d26d51af None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:04.457664 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b24bec-196d-43fb-8459-49f7d26d51af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:04.490704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-e5b24bec-196d-43fb-8459-49f7d26d51af None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:04.493034 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e5b24bec-196d-43fb-8459-49f7d26d51af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:04.572751 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79582b0-2e0b-4ff0-b84b-08a4c15e01da None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:04.576623 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:04.622035 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 406/830] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:04.623519 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b24bec-196d-43fb-8459-49f7d26d51af None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:04.648670 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d77ed0-28b0-4711-bae1-edb639a5f890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:04.649915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d77ed0-28b0-4711-bae1-edb639a5f890 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:04.650206 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d77ed0-28b0-4711-bae1-edb639a5f890 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:04.650490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d77ed0-28b0-4711-bae1-edb639a5f890 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:04.651317 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d77ed0-28b0-4711-bae1-edb639a5f890 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:04.651637 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d77ed0-28b0-4711-bae1-edb639a5f890 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:04.653192 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d77ed0-28b0-4711-bae1-edb639a5f890 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:04.656482 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:04.668329 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 425/831] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:26:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:04.670921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d77ed0-28b0-4711-bae1-edb639a5f890 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:04.797055 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b24bec-196d-43fb-8459-49f7d26d51af None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:04.833923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d77ed0-28b0-4711-bae1-edb639a5f890 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:04.858166 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c82e79-d874-4853-84d6-2a8408df182c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:04.859464 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c82e79-d874-4853-84d6-2a8408df182c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:04.859967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c82e79-d874-4853-84d6-2a8408df182c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:04.860411 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c82e79-d874-4853-84d6-2a8408df182c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:04.861548 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c82e79-d874-4853-84d6-2a8408df182c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:04.864164 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c82e79-d874-4853-84d6-2a8408df182c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:04.885767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-51c82e79-d874-4853-84d6-2a8408df182c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 08:26:04.885767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:04.885767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:26:04.885767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:26:04.885767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:26:04.885767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:26:04.885767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:26:04.885767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:04.885767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 08:26:04.885767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:04.885767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:04.885767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:04.885767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:04.885767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:04.885767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:04.885767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:04.885767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:04.886899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:04.886899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:04.886899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:04.886899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:04.886899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:04.886899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:04.886899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:26:04.886899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:26:04.886899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:26:04.886899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:26:04.886899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:04.886899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:04.886899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:04.886899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:04.886899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:04.886899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:04.887809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:04.887809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:04.887809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:04.887809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:04.887809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:04.887809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:04.887809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:04.887809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:04.887809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:04.887809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:04.887809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:26:04.887809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:04.887809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 08:26:04.887809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:04.887809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 407/832] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:26:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:04.888734 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c82e79-d874-4853-84d6-2a8408df182c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:04.896392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca555940-e9dd-4adc-bd1b-953e935ec914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:04.897216 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca555940-e9dd-4adc-bd1b-953e935ec914 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:04.897561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca555940-e9dd-4adc-bd1b-953e935ec914 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:04.897953 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca555940-e9dd-4adc-bd1b-953e935ec914 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:04.898823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca555940-e9dd-4adc-bd1b-953e935ec914 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:04.899225 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca555940-e9dd-4adc-bd1b-953e935ec914 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:04.900800 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca555940-e9dd-4adc-bd1b-953e935ec914 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:04.914182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 426/833] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:04.915077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca555940-e9dd-4adc-bd1b-953e935ec914 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:04.936060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:05.010145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c82e79-d874-4853-84d6-2a8408df182c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:05.018637 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:05.051001 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713c279-bf42-4b2b-8eb6-2070e4ff453b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:05.052618 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713c279-bf42-4b2b-8eb6-2070e4ff453b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:05.052908 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713c279-bf42-4b2b-8eb6-2070e4ff453b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:05.053187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713c279-bf42-4b2b-8eb6-2070e4ff453b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:05.064527 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713c279-bf42-4b2b-8eb6-2070e4ff453b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:26:05.069456 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca555940-e9dd-4adc-bd1b-953e935ec914 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:05.073016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-3713c279-bf42-4b2b-8eb6-2070e4ff453b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:26:05.073016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:05.073016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:05.073016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:05.073016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:05.073016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:05.073016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:05.073016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:05.073016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:05.073016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:05.073016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:05.073016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:05.073016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:26:05.073016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:26:05.073016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:26:05.073016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:26:05.075008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:26:05.075008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:26:05.075008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:26:05.075008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:26:05.075008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:05.075008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:05.075008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:05.075008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:05.075008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:05.075008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:05.075008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:05.075008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:05.075008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:05.075008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:05.075008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:05.075008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:05.077047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:05.077047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:05.077047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:05.077047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:05.077047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:26:05.077047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:26:05.077047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:26:05.077047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:26:05.077047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:26:05.077047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:26:05.077047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:26:05.077047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:05.077047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 408/834] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:26:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:05.077047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713c279-bf42-4b2b-8eb6-2070e4ff453b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:05.115377 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1861b2f-b8c5-43cd-9a54-4bf20800ddac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:05.116297 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1861b2f-b8c5-43cd-9a54-4bf20800ddac None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:05.116589 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1861b2f-b8c5-43cd-9a54-4bf20800ddac None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:05.116863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1861b2f-b8c5-43cd-9a54-4bf20800ddac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:05.117726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1861b2f-b8c5-43cd-9a54-4bf20800ddac None admin] RBAC: Adding query filter params (role.id=467d49b8a0d3435aaba58464cc8ef85c, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=84cba474fc7c470ea119de697b9d7ef9) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:05.118037 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1861b2f-b8c5-43cd-9a54-4bf20800ddac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:05.119501 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1861b2f-b8c5-43cd-9a54-4bf20800ddac None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:05.125949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 427/835] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:26:04 2022] GET /identity/v3/role_assignments?user.id=84cba474fc7c470ea119de697b9d7ef9&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=467d49b8a0d3435aaba58464cc8ef85c => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:05.126807 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1861b2f-b8c5-43cd-9a54-4bf20800ddac None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:05.201392 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:05.208770 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713c279-bf42-4b2b-8eb6-2070e4ff453b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:05.259472 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07060d8-df21-4967-9d70-1a3702daf856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:05.260420 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07060d8-df21-4967-9d70-1a3702daf856 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:05.260854 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07060d8-df21-4967-9d70-1a3702daf856 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:05.261333 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07060d8-df21-4967-9d70-1a3702daf856 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:05.262316 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07060d8-df21-4967-9d70-1a3702daf856 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:05.262772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07060d8-df21-4967-9d70-1a3702daf856 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:05.264579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07060d8-df21-4967-9d70-1a3702daf856 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:05.271733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 409/836] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:26:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:05.278574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07060d8-df21-4967-9d70-1a3702daf856 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:05.289385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1861b2f-b8c5-43cd-9a54-4bf20800ddac None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:05.319540 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:05.340962 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838b588-101f-4392-b4d9-b5f0ff11ee3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:05.342245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838b588-101f-4392-b4d9-b5f0ff11ee3c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:05.342245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838b588-101f-4392-b4d9-b5f0ff11ee3c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:05.342471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838b588-101f-4392-b4d9-b5f0ff11ee3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:05.352857 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838b588-101f-4392-b4d9-b5f0ff11ee3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:05.355939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838b588-101f-4392-b4d9-b5f0ff11ee3c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:05.377518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-0838b588-101f-4392-b4d9-b5f0ff11ee3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:05.377518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:05.377518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:05.377518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:05.377518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:05.377518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:05.377518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:05.377518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:05.377518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:05.377518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:05.377518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:05.377518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:05.377518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:26:05.377518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:26:05.377518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:26:05.379781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:26:05.379781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:05.379781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:05.379781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:05.379781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:05.379781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:05.379781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:05.379781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:05.379781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:05.379781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:05.379781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:05.379781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:05.379781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:05.379781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:05.379781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:05.379781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:05.379781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:05.382350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:26:05.382350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:26:05.382350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:26:05.382350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:26:05.382350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:26:05.382350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:26:05.382350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:05.382350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:05.382350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 428/837] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:26:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:05.382350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838b588-101f-4392-b4d9-b5f0ff11ee3c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:05.444630 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07060d8-df21-4967-9d70-1a3702daf856 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:05.498788 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838b588-101f-4392-b4d9-b5f0ff11ee3c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:05.503114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dcffa-4855-4a92-9439-3987da004690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:05.503819 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dcffa-4855-4a92-9439-3987da004690 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:05.504100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dcffa-4855-4a92-9439-3987da004690 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:05.504374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dcffa-4855-4a92-9439-3987da004690 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:05.506065 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73dcffa-4855-4a92-9439-3987da004690 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:05.506376 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73dcffa-4855-4a92-9439-3987da004690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:05.507936 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73dcffa-4855-4a92-9439-3987da004690 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:05.516711 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 410/838] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:26:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:05.517805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dcffa-4855-4a92-9439-3987da004690 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:05.545779 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b5967-b292-41ea-a4e1-98ba618abf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:05.546823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b5967-b292-41ea-a4e1-98ba618abf3b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:05.547310 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b5967-b292-41ea-a4e1-98ba618abf3b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:05.547853 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:05.548327 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b5967-b292-41ea-a4e1-98ba618abf3b None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:05.549632 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b5967-b292-41ea-a4e1-98ba618abf3b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:05.550134 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b5967-b292-41ea-a4e1-98ba618abf3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:05.551757 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b5967-b292-41ea-a4e1-98ba618abf3b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:05.558244 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 429/839] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:26:05 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:05.559336 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b5967-b292-41ea-a4e1-98ba618abf3b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:05.656079 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:05.658696 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dcffa-4855-4a92-9439-3987da004690 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:05.705024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005a3b7-83e7-47d4-b7f8-36bf491db163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:05.705882 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005a3b7-83e7-47d4-b7f8-36bf491db163 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:05.706515 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005a3b7-83e7-47d4-b7f8-36bf491db163 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:05.706515 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005a3b7-83e7-47d4-b7f8-36bf491db163 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:05.707534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005a3b7-83e7-47d4-b7f8-36bf491db163 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:05.709268 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005a3b7-83e7-47d4-b7f8-36bf491db163 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:05.711047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005a3b7-83e7-47d4-b7f8-36bf491db163 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:05.726630 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 411/840] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:26:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:05.735335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005a3b7-83e7-47d4-b7f8-36bf491db163 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:05.751610 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b5967-b292-41ea-a4e1-98ba618abf3b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:05.799368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0e0d5-98f6-43ee-ab15-0a375de0f4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:05.800334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0e0d5-98f6-43ee-ab15-0a375de0f4eb None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:05.800620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0e0d5-98f6-43ee-ab15-0a375de0f4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:05.800901 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0e0d5-98f6-43ee-ab15-0a375de0f4eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:05.815155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b0e0d5-98f6-43ee-ab15-0a375de0f4eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:05.817791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b0e0d5-98f6-43ee-ab15-0a375de0f4eb None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:05.848255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-41b0e0d5-98f6-43ee-ab15-0a375de0f4eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:05.848255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:05.848255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:05.848255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:05.848255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:05.848255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:05.848255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:05.848255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:05.848255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:05.848255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:05.848255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:05.848255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:05.848255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:05.848255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:05.848255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:05.850463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:05.850463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:05.850463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:05.850463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:05.850463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:05.850463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:05.850463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:05.850463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:05.850463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:05.850463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:05.850463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:05.850463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:05.850463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:05.850463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:05.850463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:05.850463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:05.850463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:05.851458 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:05.851458 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:05.851458 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:05.851458 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:05.855722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 430/841] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:05.860332 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0e0d5-98f6-43ee-ab15-0a375de0f4eb None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:05.888912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:05.935239 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005a3b7-83e7-47d4-b7f8-36bf491db163 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:05.982439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b0863-39fd-46c0-90b4-d77f4e6548c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:05.984151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b0863-39fd-46c0-90b4-d77f4e6548c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:05.984618 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b0863-39fd-46c0-90b4-d77f4e6548c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:05.985099 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b0863-39fd-46c0-90b4-d77f4e6548c7 None admin] PATH_INFO: `/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/2b03f8b8c51e472bbd4cf0afc9c6b632/roles/9fc4c3dcb90945ebb4eb5566e62e7833` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:06.007539 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b0863-39fd-46c0-90b4-d77f4e6548c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=35c846931fd443fbb83a6fbe0aeaa866, user_id=2b03f8b8c51e472bbd4cf0afc9c6b632, role_id=9fc4c3dcb90945ebb4eb5566e62e7833)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:06.009316 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b0863-39fd-46c0-90b4-d77f4e6548c7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:06.025585 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:06.037652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0e0d5-98f6-43ee-ab15-0a375de0f4eb None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:06.083466 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 412/842] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:26:05 2022] PUT /identity/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/2b03f8b8c51e472bbd4cf0afc9c6b632/roles/9fc4c3dcb90945ebb4eb5566e62e7833 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:26:06.103993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cf765-323a-45a7-b007-df834d8cda84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:06.104699 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cf765-323a-45a7-b007-df834d8cda84 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:06.104981 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cf765-323a-45a7-b007-df834d8cda84 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:06.105260 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cf765-323a-45a7-b007-df834d8cda84 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:06.106036 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12cf765-323a-45a7-b007-df834d8cda84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:06.106368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12cf765-323a-45a7-b007-df834d8cda84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:06.107927 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12cf765-323a-45a7-b007-df834d8cda84 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:06.124739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 431/843] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:06.130720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b0863-39fd-46c0-90b4-d77f4e6548c7 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:06.264144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b0863-39fd-46c0-90b4-d77f4e6548c7 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:06.307422 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:06.313870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c6609-dc45-4dad-90e8-c3814321b463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:06.315084 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c6609-dc45-4dad-90e8-c3814321b463 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:06.315889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c6609-dc45-4dad-90e8-c3814321b463 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:06.316387 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c6609-dc45-4dad-90e8-c3814321b463 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:06.325661 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c6609-dc45-4dad-90e8-c3814321b463 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:06.327528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c6609-dc45-4dad-90e8-c3814321b463 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:06.347062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-6c3c6609-dc45-4dad-90e8-c3814321b463 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 08:26:06.347062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:06.347062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:06.347062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:06.347062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:06.347062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:06.347062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:06.347062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:06.347062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:06.347062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:06.347062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:06.347062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:06.347062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:26:06.347062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:26:06.347062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:26:06.347062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:26:06.348140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:06.348140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:06.348140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:26:06.348140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:06.348140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:26:06.348140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:06.348140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:06.348140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:06.348140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:06.348140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:06.348140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:06.348140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:06.348140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:06.348140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:06.348140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:06.348140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:06.348140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:06.348140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:06.349710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:06.349710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:06.349710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:26:06.349710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:26:06.349710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:26:06.349710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:26:06.349710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:26:06.349710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:26:06.349710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 08:26:06.349710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:06.352832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 413/844] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:26:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:06.359405 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cf765-323a-45a7-b007-df834d8cda84 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:06.490652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cf765-323a-45a7-b007-df834d8cda84 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:06.536490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a5020-0d02-4fd2-8107-9d6f8b92790b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:06.536490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a5020-0d02-4fd2-8107-9d6f8b92790b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:06.536490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a5020-0d02-4fd2-8107-9d6f8b92790b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:06.536490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a5020-0d02-4fd2-8107-9d6f8b92790b None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:06.536933 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a5020-0d02-4fd2-8107-9d6f8b92790b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:06.536933 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a5020-0d02-4fd2-8107-9d6f8b92790b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:06.536933 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a5020-0d02-4fd2-8107-9d6f8b92790b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:06.557937 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 432/845] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Jul 18 08:26:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:06.564836 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c6609-dc45-4dad-90e8-c3814321b463 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:06.689160 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c6609-dc45-4dad-90e8-c3814321b463 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:06.728999 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9ec51-17f0-4cff-b933-e5922870be2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:06.729658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9ec51-17f0-4cff-b933-e5922870be2a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:06.729942 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9ec51-17f0-4cff-b933-e5922870be2a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:06.730215 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9ec51-17f0-4cff-b933-e5922870be2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:06.799566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:06.812191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9ec51-17f0-4cff-b933-e5922870be2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:06.813732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9ec51-17f0-4cff-b933-e5922870be2a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:06.930141 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 414/846] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:26:06 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 08:26:06.935893 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a5020-0d02-4fd2-8107-9d6f8b92790b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:06.955732 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:07.057281 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a5020-0d02-4fd2-8107-9d6f8b92790b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:07.105379 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29450830-7c51-45a9-b15c-396a8a4eb127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:07.107256 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29450830-7c51-45a9-b15c-396a8a4eb127 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:07.108178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29450830-7c51-45a9-b15c-396a8a4eb127 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:07.109011 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29450830-7c51-45a9-b15c-396a8a4eb127 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:07.130905 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29450830-7c51-45a9-b15c-396a8a4eb127 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:07.134089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29450830-7c51-45a9-b15c-396a8a4eb127 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:07.170396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-29450830-7c51-45a9-b15c-396a8a4eb127 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:07.170396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:07.170396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:07.170396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:07.170396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:07.170396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:07.170396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:07.170396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:07.170396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:07.170396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:07.170396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:07.170396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:07.170396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:07.170396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:07.170396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:07.172152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:07.172152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:07.172152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:07.172152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:07.172152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:07.172152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:07.172152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:07.172152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:07.172152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:07.172152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:07.172152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:07.172152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:07.172152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:07.172152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:07.172152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:07.172152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:07.172152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:07.173944 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:07.173944 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:07.173944 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:07.173944 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:07.178479 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 433/847] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:07.188131 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9ec51-17f0-4cff-b933-e5922870be2a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:07.317603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:07.328446 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9ec51-17f0-4cff-b933-e5922870be2a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:07.370799 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae5a49-239a-45bf-bb7e-52b92bef1cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:07.371851 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae5a49-239a-45bf-bb7e-52b92bef1cdb None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:07.372331 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae5a49-239a-45bf-bb7e-52b92bef1cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:07.372798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae5a49-239a-45bf-bb7e-52b92bef1cdb None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:07.373763 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae5a49-239a-45bf-bb7e-52b92bef1cdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:07.374261 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae5a49-239a-45bf-bb7e-52b92bef1cdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:07.375871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae5a49-239a-45bf-bb7e-52b92bef1cdb None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:07.390710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 415/848] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:07.398343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29450830-7c51-45a9-b15c-396a8a4eb127 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:07.432972 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f13f0e-dc0e-4082-9874-3ce8d18e11ad None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:07.433314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f13f0e-dc0e-4082-9874-3ce8d18e11ad None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:07.433540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f13f0e-dc0e-4082-9874-3ce8d18e11ad None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:07.435141 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 416/849] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:07.440991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3854e-806b-41cc-a136-aaf08304f7ef None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:07.441307 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3854e-806b-41cc-a136-aaf08304f7ef None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:07.441560 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3854e-806b-41cc-a136-aaf08304f7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:07.465832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-55d3854e-806b-41cc-a136-aaf08304f7ef None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:07.467718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-55d3854e-806b-41cc-a136-aaf08304f7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:07.491417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:07.550328 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29450830-7c51-45a9-b15c-396a8a4eb127 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:07.599743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f2d808-dc9f-4890-9eab-9f2497103cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:07.600837 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2d808-dc9f-4890-9eab-9f2497103cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:07.601277 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2d808-dc9f-4890-9eab-9f2497103cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:07.601693 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2d808-dc9f-4890-9eab-9f2497103cb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:07.611655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f2d808-dc9f-4890-9eab-9f2497103cb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:07.614592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f2d808-dc9f-4890-9eab-9f2497103cb1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:07.631847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 417/850] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:07.632730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-03f2d808-dc9f-4890-9eab-9f2497103cb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:07.632730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:07.632730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:07.632730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:07.632730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:07.632730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:07.632730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:07.632730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:07.632730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:07.632730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:07.632730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:07.632730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:07.632730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:26:07.632730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:26:07.632730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:26:07.633618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:26:07.633618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:07.633618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:07.633618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:07.633618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:07.633618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:07.633618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:07.633618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:07.633618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:07.633618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:07.633618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:07.633618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:07.633618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:07.633618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:07.633618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:07.633618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:07.633618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:07.634522 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:26:07.634522 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:26:07.634522 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:26:07.634522 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:26:07.634522 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:26:07.634522 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:26:07.634522 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:07.634522 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:07.636802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 434/851] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:26:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:07.645193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3854e-806b-41cc-a136-aaf08304f7ef None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:07.669458 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f2d808-dc9f-4890-9eab-9f2497103cb1 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:07.808204 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:07.839344 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3854e-806b-41cc-a136-aaf08304f7ef None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:07.868406 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f2d808-dc9f-4890-9eab-9f2497103cb1 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:07.888785 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43406f-8f45-45f1-a5fa-5560c3915315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:07.889468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43406f-8f45-45f1-a5fa-5560c3915315 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:07.889790 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43406f-8f45-45f1-a5fa-5560c3915315 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:07.890058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43406f-8f45-45f1-a5fa-5560c3915315 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:07.890858 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43406f-8f45-45f1-a5fa-5560c3915315 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:07.891170 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43406f-8f45-45f1-a5fa-5560c3915315 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:07.892662 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43406f-8f45-45f1-a5fa-5560c3915315 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:07.911967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 418/852] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:26:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:07.913510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9ce2b-4011-40ca-9375-ba91f8bd338a None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:07.913800 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9ce2b-4011-40ca-9375-ba91f8bd338a None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:07.914098 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9ce2b-4011-40ca-9375-ba91f8bd338a None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:07.914517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:07.919407 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 419/853] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:07.919407 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9ce2b-4011-40ca-9375-ba91f8bd338a None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:07.929656 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e80ff2-2e21-4c18-a08d-24be9b9e2a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:07.930569 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e80ff2-2e21-4c18-a08d-24be9b9e2a8e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:07.933751 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e80ff2-2e21-4c18-a08d-24be9b9e2a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:07.933751 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e80ff2-2e21-4c18-a08d-24be9b9e2a8e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:07.933751 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e80ff2-2e21-4c18-a08d-24be9b9e2a8e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:07.933751 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e80ff2-2e21-4c18-a08d-24be9b9e2a8e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:07.959457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-13e80ff2-2e21-4c18-a08d-24be9b9e2a8e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 08:26:07.959457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:07.959457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:26:07.959457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:26:07.959457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:26:07.959457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:26:07.959457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:26:07.959457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:07.959457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 08:26:07.959457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:07.959457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:07.959457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:07.959457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:07.959457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:07.959457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:07.959457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:07.960606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:07.960606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:07.960606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:07.960606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:07.960606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:07.960606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:07.960606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:07.960606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:26:07.960606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:26:07.960606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:26:07.960606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:26:07.960606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:07.960606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:07.960606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:07.960606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:07.960606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:07.961445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:07.961445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:07.961445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:07.961445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:07.961445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:07.961445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:07.961445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:07.961445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:07.961445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:07.961445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:07.961445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:07.961445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:26:07.961445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:07.961445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 08:26:07.961445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:07.970534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 435/854] 10.222.0.57 () {56 vars in 1132 bytes} [Mon Jul 18 08:26:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:07.970534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b0744-140f-4bb8-8145-e3cb164ba186 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:07.970534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b0744-140f-4bb8-8145-e3cb164ba186 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:07.970534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b0744-140f-4bb8-8145-e3cb164ba186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:07.998401 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0d2b0744-140f-4bb8-8145-e3cb164ba186 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:08.000508 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0d2b0744-140f-4bb8-8145-e3cb164ba186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:08.049725 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9ce2b-4011-40ca-9375-ba91f8bd338a None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:08.086510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa1737-f0b4-42cb-9f17-940e4b5571b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:08.087186 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa1737-f0b4-42cb-9f17-940e4b5571b1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:08.087474 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa1737-f0b4-42cb-9f17-940e4b5571b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:08.087784 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa1737-f0b4-42cb-9f17-940e4b5571b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:08.088635 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa1737-f0b4-42cb-9f17-940e4b5571b1 None admin] RBAC: Adding query filter params (role.id=467d49b8a0d3435aaba58464cc8ef85c, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=09a5c2e804504038817cad76544dce5f) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:08.088952 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa1737-f0b4-42cb-9f17-940e4b5571b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:08.090402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa1737-f0b4-42cb-9f17-940e4b5571b1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:08.096987 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 420/855] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:26:07 2022] GET /identity/v3/role_assignments?user.id=09a5c2e804504038817cad76544dce5f&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=467d49b8a0d3435aaba58464cc8ef85c => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:08.097578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa1737-f0b4-42cb-9f17-940e4b5571b1 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:08.119458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:08.134890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 436/856] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:08.159639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b0744-140f-4bb8-8145-e3cb164ba186 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:08.309577 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa1737-f0b4-42cb-9f17-940e4b5571b1 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:08.334470 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b0744-140f-4bb8-8145-e3cb164ba186 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:08.361929 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:08.379336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e9d5e-3e40-475b-b140-cd70388efbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:08.381021 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e9d5e-3e40-475b-b140-cd70388efbf6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:08.381543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e9d5e-3e40-475b-b140-cd70388efbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:08.381998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e9d5e-3e40-475b-b140-cd70388efbf6 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:08.382769 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e9d5e-3e40-475b-b140-cd70388efbf6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:08.383082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e9d5e-3e40-475b-b140-cd70388efbf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:08.384549 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e9d5e-3e40-475b-b140-cd70388efbf6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:08.390142 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c268c-6133-4b92-a77a-41ea38e2d911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:08.391925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 421/857] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 18 08:26:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:08.392312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c268c-6133-4b92-a77a-41ea38e2d911 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:08.393184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c268c-6133-4b92-a77a-41ea38e2d911 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:08.393928 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c268c-6133-4b92-a77a-41ea38e2d911 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:08.395723 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c268c-6133-4b92-a77a-41ea38e2d911 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:08.398664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c268c-6133-4b92-a77a-41ea38e2d911 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:08.399593 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e9d5e-3e40-475b-b140-cd70388efbf6 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:08.488535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:08.511146 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 437/858] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 18 08:26:08 2022] POST /identity/v3/services => generated 251 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:26:08.545622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e9d5e-3e40-475b-b140-cd70388efbf6 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:08.588180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e50d1d-abf8-4d3f-99d1-0e732baffd82 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:08.588529 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e50d1d-abf8-4d3f-99d1-0e732baffd82 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:08.588753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e50d1d-abf8-4d3f-99d1-0e732baffd82 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:08.590500 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 438/859] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:26:08.596377 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ab619-c096-4389-8334-50f6bf206e7b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:08.600396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ab619-c096-4389-8334-50f6bf206e7b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:08.600396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ab619-c096-4389-8334-50f6bf206e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:08.601092 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fc196-5235-470b-a1d5-32b607df2334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:08.602129 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fc196-5235-470b-a1d5-32b607df2334 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:08.602689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fc196-5235-470b-a1d5-32b607df2334 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:08.603455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fc196-5235-470b-a1d5-32b607df2334 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:08.605477 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fc196-5235-470b-a1d5-32b607df2334 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:08.606011 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fc196-5235-470b-a1d5-32b607df2334 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:08.607752 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fc196-5235-470b-a1d5-32b607df2334 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:08.615199 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 422/860] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 18 08:26:08 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:08.621156 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-185ab619-c096-4389-8334-50f6bf206e7b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:08.622969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-185ab619-c096-4389-8334-50f6bf206e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:08.710256 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 439/861] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:08 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:08.734798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fc196-5235-470b-a1d5-32b607df2334 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:08.865276 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fc196-5235-470b-a1d5-32b607df2334 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:08.898410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:08.905465 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625df93b-88d5-4afe-844b-870b0b76c46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:08.906270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625df93b-88d5-4afe-844b-870b0b76c46d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:08.906548 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625df93b-88d5-4afe-844b-870b0b76c46d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:08.906815 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625df93b-88d5-4afe-844b-870b0b76c46d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:08.919365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625df93b-88d5-4afe-844b-870b0b76c46d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:26:08.927379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-625df93b-88d5-4afe-844b-870b0b76c46d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:26:08.927379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:08.927379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:08.927379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:08.927379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:08.927379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:08.927379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:08.927379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:08.927379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:08.927379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:08.927379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:08.927379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:08.927379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:26:08.927379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:26:08.927379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:26:08.927379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:26:08.929496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:26:08.929496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:26:08.929496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:26:08.929496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:26:08.929496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:08.929496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:08.929496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:08.929496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:08.929496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:08.929496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:08.929496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:08.929496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:08.929496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:08.929496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:08.929496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:08.929496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:08.931585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:08.931585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:08.931585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:08.931585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:08.931585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:26:08.931585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:26:08.931585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:26:08.931585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:26:08.931585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:26:08.931585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:26:08.931585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:26:08.931585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:08.931585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 423/862] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:26:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:08.938329 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ab619-c096-4389-8334-50f6bf206e7b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:09.060049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ab619-c096-4389-8334-50f6bf206e7b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:09.073398 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:09.098699 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6508224-f128-4296-9975-b9fe4849712c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:09.098699 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6508224-f128-4296-9975-b9fe4849712c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:09.099874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6508224-f128-4296-9975-b9fe4849712c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:09.100057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6508224-f128-4296-9975-b9fe4849712c None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:09.100800 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6508224-f128-4296-9975-b9fe4849712c None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:09.101115 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6508224-f128-4296-9975-b9fe4849712c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:09.102530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6508224-f128-4296-9975-b9fe4849712c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:09.112303 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 440/863] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:26:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:09.123398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625df93b-88d5-4afe-844b-870b0b76c46d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:09.273721 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625df93b-88d5-4afe-844b-870b0b76c46d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:09.324988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca29f908-32da-4b74-bb8f-54b50c730c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:09.326608 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29f908-32da-4b74-bb8f-54b50c730c0b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:09.327110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29f908-32da-4b74-bb8f-54b50c730c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:09.327617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29f908-32da-4b74-bb8f-54b50c730c0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:09.340989 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca29f908-32da-4b74-bb8f-54b50c730c0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:09.342856 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca29f908-32da-4b74-bb8f-54b50c730c0b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:09.365148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-ca29f908-32da-4b74-bb8f-54b50c730c0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:09.365148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:09.365148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:09.365148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:09.365148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:09.365148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:09.365148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:09.365148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:09.365148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:09.365148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:09.365148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:09.365148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:09.365148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:09.365148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:09.365148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:09.366248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:09.366248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:09.366248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:09.366248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:09.366248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:09.366248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:09.366248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:09.366248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:09.366248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:09.366248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:09.366248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:09.366248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:09.366248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:09.366248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:09.366248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:09.366248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:09.366248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:09.367151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:09.367151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:09.367151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:09.367151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:09.369652 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 424/864] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:09.375804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6508224-f128-4296-9975-b9fe4849712c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:09.522688 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6508224-f128-4296-9975-b9fe4849712c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:09.563290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38c105-4289-4496-aad6-ef21390da2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:09.564014 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38c105-4289-4496-aad6-ef21390da2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:09.564304 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38c105-4289-4496-aad6-ef21390da2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:09.564586 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38c105-4289-4496-aad6-ef21390da2f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:09.565331 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c38c105-4289-4496-aad6-ef21390da2f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:09.565643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c38c105-4289-4496-aad6-ef21390da2f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:09.567652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c38c105-4289-4496-aad6-ef21390da2f5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:09.583167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 441/865] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:09.588221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca29f908-32da-4b74-bb8f-54b50c730c0b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:09.647346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:09.734719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca29f908-32da-4b74-bb8f-54b50c730c0b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:09.794519 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1064f-855d-4b01-8b49-b062a5e6fbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:09.795967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1064f-855d-4b01-8b49-b062a5e6fbc3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:09.796468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1064f-855d-4b01-8b49-b062a5e6fbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:09.796818 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1064f-855d-4b01-8b49-b062a5e6fbc3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:09.810058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1064f-855d-4b01-8b49-b062a5e6fbc3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:09.812796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1064f-855d-4b01-8b49-b062a5e6fbc3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:09.841941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-abc1064f-855d-4b01-8b49-b062a5e6fbc3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 08:26:09.841941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:09.841941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:09.841941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:09.841941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:09.841941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:09.841941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:09.841941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:09.841941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:09.841941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:09.841941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:09.841941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:09.841941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:26:09.841941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:26:09.841941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:26:09.841941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:26:09.842903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:09.842903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:09.842903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:26:09.842903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:09.842903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:26:09.842903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:09.842903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:09.842903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:09.842903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:09.842903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:09.842903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:09.842903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:09.842903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:09.842903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:09.842903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:09.842903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:09.842903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:09.842903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:09.843881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:09.843881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:09.843881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:26:09.843881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:26:09.843881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:26:09.843881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:26:09.843881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:26:09.843881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:26:09.843881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 08:26:09.843881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:09.844900 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 425/866] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 18 08:26:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:09.850139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38c105-4289-4496-aad6-ef21390da2f5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:09.891453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:10.012179 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38c105-4289-4496-aad6-ef21390da2f5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:10.072582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e2966-f5bb-421f-8efe-838a55a04d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:10.073991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e2966-f5bb-421f-8efe-838a55a04d64 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:10.074705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e2966-f5bb-421f-8efe-838a55a04d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:10.075355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e2966-f5bb-421f-8efe-838a55a04d64 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:10.077192 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e2966-f5bb-421f-8efe-838a55a04d64 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:10.077965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e2966-f5bb-421f-8efe-838a55a04d64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:10.080790 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e2966-f5bb-421f-8efe-838a55a04d64 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:10.117870 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 442/867] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Jul 18 08:26:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:10.126904 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1064f-855d-4b01-8b49-b062a5e6fbc3 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:10.274155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1064f-855d-4b01-8b49-b062a5e6fbc3 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:10.276824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:10.320007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ec8ec-1f08-49a9-8afb-8ad6f5fdebed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:10.320677 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ec8ec-1f08-49a9-8afb-8ad6f5fdebed None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:10.320965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ec8ec-1f08-49a9-8afb-8ad6f5fdebed None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:10.321236 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ec8ec-1f08-49a9-8afb-8ad6f5fdebed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:10.398176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ec8ec-1f08-49a9-8afb-8ad6f5fdebed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:10.399532 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ec8ec-1f08-49a9-8afb-8ad6f5fdebed None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:10.513243 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 426/868] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:26:10 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 08:26:10.523112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e2966-f5bb-421f-8efe-838a55a04d64 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:10.588073 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b548e6-2136-4f2c-a117-b69157d58d1b None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:10.588410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b548e6-2136-4f2c-a117-b69157d58d1b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:10.588625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b548e6-2136-4f2c-a117-b69157d58d1b None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:10.590274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 427/869] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:10.595826 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c22561-39be-4ada-932b-80ae9b8e7a04 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:10.596140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c22561-39be-4ada-932b-80ae9b8e7a04 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:10.596382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c22561-39be-4ada-932b-80ae9b8e7a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:10.602354 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:10.616228 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-32c22561-39be-4ada-932b-80ae9b8e7a04 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:10.617786 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-32c22561-39be-4ada-932b-80ae9b8e7a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:10.643971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:10.656312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e2966-f5bb-421f-8efe-838a55a04d64 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:10.698987 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a47c6e-de14-4c12-a9de-6db3cb839db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:10.699892 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a47c6e-de14-4c12-a9de-6db3cb839db5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:10.700180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a47c6e-de14-4c12-a9de-6db3cb839db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:10.700458 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a47c6e-de14-4c12-a9de-6db3cb839db5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:10.713514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a47c6e-de14-4c12-a9de-6db3cb839db5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:10.715019 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a47c6e-de14-4c12-a9de-6db3cb839db5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:10.735639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-54a47c6e-de14-4c12-a9de-6db3cb839db5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:10.735639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:10.735639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:10.735639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:10.735639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:10.735639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:10.735639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:10.735639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:10.735639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:10.735639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:10.735639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:10.735639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:10.735639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:10.735639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:10.735639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:10.737365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:10.737365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:10.737365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:10.737365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:10.737365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:10.737365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:10.737365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:10.737365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:10.737365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:10.737365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:10.737365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:10.737365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:10.737365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:10.737365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:10.737365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:10.737365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:10.737365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:10.739368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:10.739368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:10.739368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:10.739368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:10.741065 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 428/871] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:10 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:10.742262 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 443/871] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:10.743321 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c22561-39be-4ada-932b-80ae9b8e7a04 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:10.763903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a47c6e-de14-4c12-a9de-6db3cb839db5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:10.894251 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a47c6e-de14-4c12-a9de-6db3cb839db5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:10.930260 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c22561-39be-4ada-932b-80ae9b8e7a04 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:10.937780 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c3b12-7f81-46e2-b4a5-9f985768f20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:10.939058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c3b12-7f81-46e2-b4a5-9f985768f20c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:10.939512 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c3b12-7f81-46e2-b4a5-9f985768f20c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:10.940007 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c3b12-7f81-46e2-b4a5-9f985768f20c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:10.941163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c3b12-7f81-46e2-b4a5-9f985768f20c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:10.944199 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c3b12-7f81-46e2-b4a5-9f985768f20c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:10.966312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-977c3b12-7f81-46e2-b4a5-9f985768f20c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 08:26:10.966312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:10.966312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:26:10.966312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:26:10.966312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:26:10.966312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:26:10.966312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:26:10.966312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:10.966312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 08:26:10.966312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:10.966312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:10.966312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:10.966312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:10.966312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:10.966312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:10.966312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:10.966312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:10.968211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:10.968211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:10.968211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:10.968211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:10.968211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:10.968211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:10.968211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:26:10.968211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:26:10.968211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:26:10.968211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:26:10.968211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:10.968211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:10.968211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:10.968211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:10.968211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:10.968211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:10.969938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:10.969938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:10.969938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:10.969938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:10.969938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:10.969938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:10.969938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:10.969938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:10.969938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:10.969938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:10.969938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:26:10.969938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:10.969938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 08:26:10.969938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:10.975599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 444/872] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 18 08:26:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:10.976077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24024678-6a4e-4326-9f9c-3a0a6e8cf8d4 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:10.976561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24024678-6a4e-4326-9f9c-3a0a6e8cf8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:10.976980 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24024678-6a4e-4326-9f9c-3a0a6e8cf8d4 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:10.978895 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 445/873] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:10.979906 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24024678-6a4e-4326-9f9c-3a0a6e8cf8d4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:10.994371 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:10.996449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:11.000267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57f992-58cb-4b10-ab89-0216a479e156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:11.001176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57f992-58cb-4b10-ab89-0216a479e156 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:11.001466 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57f992-58cb-4b10-ab89-0216a479e156 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:11.001910 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57f992-58cb-4b10-ab89-0216a479e156 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:11.003129 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57f992-58cb-4b10-ab89-0216a479e156 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:11.003630 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57f992-58cb-4b10-ab89-0216a479e156 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:11.008889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57f992-58cb-4b10-ab89-0216a479e156 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:11.032550 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 429/874] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:11.032550 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3e5e4-dad9-446c-a36d-cc60b3dde315 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:11.032550 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3e5e4-dad9-446c-a36d-cc60b3dde315 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:11.032550 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3e5e4-dad9-446c-a36d-cc60b3dde315 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:11.037658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 430/875] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:11.037658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf945b-db7b-4995-bb78-03f5fc4213d0 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:11.037658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf945b-db7b-4995-bb78-03f5fc4213d0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:11.037658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf945b-db7b-4995-bb78-03f5fc4213d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:11.059325 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-dbdf945b-db7b-4995-bb78-03f5fc4213d0 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:11.060898 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-dbdf945b-db7b-4995-bb78-03f5fc4213d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:11.137091 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24024678-6a4e-4326-9f9c-3a0a6e8cf8d4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:11.153793 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 431/876] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:11 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:11.155000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf945b-db7b-4995-bb78-03f5fc4213d0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:11.198420 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71fe6d7-1422-4159-a8ef-aad40e5058b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:11.199429 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fe6d7-1422-4159-a8ef-aad40e5058b9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:11.199932 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fe6d7-1422-4159-a8ef-aad40e5058b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:11.200377 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fe6d7-1422-4159-a8ef-aad40e5058b9 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:11.201562 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71fe6d7-1422-4159-a8ef-aad40e5058b9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:11.201983 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71fe6d7-1422-4159-a8ef-aad40e5058b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:11.204382 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71fe6d7-1422-4159-a8ef-aad40e5058b9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:11.212941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 446/877] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 18 08:26:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:11.214377 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7b2d4-a8b6-42ff-b98b-c59ef5b8df13 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:11.214670 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7b2d4-a8b6-42ff-b98b-c59ef5b8df13 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:11.214951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7b2d4-a8b6-42ff-b98b-c59ef5b8df13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:11.232432 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-42a7b2d4-a8b6-42ff-b98b-c59ef5b8df13 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:11.233635 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-42a7b2d4-a8b6-42ff-b98b-c59ef5b8df13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:11.301669 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf945b-db7b-4995-bb78-03f5fc4213d0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:11.316016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:11.322510 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 447/878] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:11 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:11.323491 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7b2d4-a8b6-42ff-b98b-c59ef5b8df13 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:11.334748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:11.352180 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517a268-5cab-463b-88b7-a9931c91046c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:11.353215 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517a268-5cab-463b-88b7-a9931c91046c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:11.353624 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517a268-5cab-463b-88b7-a9931c91046c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:11.354038 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517a268-5cab-463b-88b7-a9931c91046c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:11.364179 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517a268-5cab-463b-88b7-a9931c91046c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:11.366112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517a268-5cab-463b-88b7-a9931c91046c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:11.382796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-2517a268-5cab-463b-88b7-a9931c91046c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:11.382796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:11.382796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:11.382796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:11.382796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:11.382796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:11.382796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:11.382796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:11.382796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:11.382796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:11.382796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:11.382796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:11.382796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:26:11.382796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:26:11.382796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:26:11.384798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:26:11.384798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:11.384798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:11.384798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:11.384798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:11.384798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:11.384798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:11.384798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:11.384798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:11.384798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:11.384798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:11.384798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:11.384798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:11.384798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:11.384798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:11.384798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:11.384798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:11.387035 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:26:11.387035 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:26:11.387035 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:26:11.387035 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:26:11.387035 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:26:11.387035 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:26:11.387035 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:11.387035 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:11.388347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 432/879] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:26:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:11.390143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517a268-5cab-463b-88b7-a9931c91046c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:11.467584 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7b2d4-a8b6-42ff-b98b-c59ef5b8df13 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:11.525218 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c6768-5116-4648-a796-c30881be6b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:11.526502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c6768-5116-4648-a796-c30881be6b1f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:11.526973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c6768-5116-4648-a796-c30881be6b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:11.527363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c6768-5116-4648-a796-c30881be6b1f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:11.528190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c6768-5116-4648-a796-c30881be6b1f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:11.529728 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c6768-5116-4648-a796-c30881be6b1f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:11.543120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-394c6768-5116-4648-a796-c30881be6b1f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 08:26:11.543120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:11.543120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:26:11.543120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:26:11.543120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:26:11.543120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:26:11.543120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:26:11.543120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:11.543120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 08:26:11.543120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:11.543120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:11.543120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:11.543120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:11.543120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:11.543120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:11.543120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:11.543120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:11.551787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:11.551787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:11.551787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:11.551787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:11.551787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:11.551787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:11.551787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:26:11.551787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:26:11.551787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:26:11.551787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:26:11.551787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:11.551787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:11.551787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:11.551787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:11.551787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:11.551787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:11.552886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:11.552886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:11.552886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:11.552886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:11.552886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:11.552886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:11.552886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:11.552886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:11.552886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:11.552886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:11.552886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:26:11.552886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:11.552886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 08:26:11.552886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:11.552886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 448/880] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:26:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:11.553793 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c6768-5116-4648-a796-c30881be6b1f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:11.553877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517a268-5cab-463b-88b7-a9931c91046c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:11.597107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfbdb3-b841-48f4-941c-183f6b659929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:11.597783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfbdb3-b841-48f4-941c-183f6b659929 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:11.598064 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfbdb3-b841-48f4-941c-183f6b659929 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:11.598338 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfbdb3-b841-48f4-941c-183f6b659929 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:11.599066 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfbdb3-b841-48f4-941c-183f6b659929 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:11.599418 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfbdb3-b841-48f4-941c-183f6b659929 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:11.600788 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfbdb3-b841-48f4-941c-183f6b659929 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:11.605677 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 433/881] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 18 08:26:11 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:11.606516 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfbdb3-b841-48f4-941c-183f6b659929 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:11.627286 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:11.675916 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:11.680159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c6768-5116-4648-a796-c30881be6b1f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:11.725988 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574419b-bd7e-422a-8550-961fbc92137b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:11.726578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574419b-bd7e-422a-8550-961fbc92137b None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:11.726863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574419b-bd7e-422a-8550-961fbc92137b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:11.727149 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574419b-bd7e-422a-8550-961fbc92137b None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:11.727944 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574419b-bd7e-422a-8550-961fbc92137b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:11.729446 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574419b-bd7e-422a-8550-961fbc92137b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:11.751015 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfbdb3-b841-48f4-941c-183f6b659929 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:11.797631 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 449/882] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 18 08:26:11 2022] POST /identity/v3/services => generated 269 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:26:11.799257 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574419b-bd7e-422a-8550-961fbc92137b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:11.808774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f896a1-ecb4-42a9-b756-18a398023987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:11.810550 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f896a1-ecb4-42a9-b756-18a398023987 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:11.810838 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f896a1-ecb4-42a9-b756-18a398023987 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:11.811123 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f896a1-ecb4-42a9-b756-18a398023987 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:11.812053 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f896a1-ecb4-42a9-b756-18a398023987 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:11.812370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f896a1-ecb4-42a9-b756-18a398023987 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:11.813887 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f896a1-ecb4-42a9-b756-18a398023987 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:11.830681 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 434/883] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:26:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:11.839577 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f896a1-ecb4-42a9-b756-18a398023987 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:11.990101 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f896a1-ecb4-42a9-b756-18a398023987 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:11.992345 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:12.034994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574419b-bd7e-422a-8550-961fbc92137b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:12.040053 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3268c-c09a-4cdf-9842-76149d23e69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:12.041147 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3268c-c09a-4cdf-9842-76149d23e69e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:12.041597 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3268c-c09a-4cdf-9842-76149d23e69e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:12.042025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3268c-c09a-4cdf-9842-76149d23e69e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:12.043457 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3268c-c09a-4cdf-9842-76149d23e69e None admin] RBAC: Adding query filter params (role.id=9fc4c3dcb90945ebb4eb5566e62e7833, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=2b03f8b8c51e472bbd4cf0afc9c6b632) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:12.043992 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3268c-c09a-4cdf-9842-76149d23e69e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:12.046283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3268c-c09a-4cdf-9842-76149d23e69e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:12.061673 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 435/884] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:26:11 2022] GET /identity/v3/role_assignments?user.id=2b03f8b8c51e472bbd4cf0afc9c6b632&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=9fc4c3dcb90945ebb4eb5566e62e7833 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:12.085071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:12.107854 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ea33d-0319-4269-8d29-10f1216096d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:12.109283 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ea33d-0319-4269-8d29-10f1216096d0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:12.109800 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ea33d-0319-4269-8d29-10f1216096d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:12.110228 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ea33d-0319-4269-8d29-10f1216096d0 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:12.111572 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ea33d-0319-4269-8d29-10f1216096d0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:12.112106 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ea33d-0319-4269-8d29-10f1216096d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:12.114915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ea33d-0319-4269-8d29-10f1216096d0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:12.124024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 450/885] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:26:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:12.130303 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3268c-c09a-4cdf-9842-76149d23e69e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:12.328007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3268c-c09a-4cdf-9842-76149d23e69e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:12.380819 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edffd1-9793-4105-9ae5-478d0a234e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:12.382000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edffd1-9793-4105-9ae5-478d0a234e1a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:12.382505 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edffd1-9793-4105-9ae5-478d0a234e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:12.383017 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edffd1-9793-4105-9ae5-478d0a234e1a None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:12.384749 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edffd1-9793-4105-9ae5-478d0a234e1a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:12.385308 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edffd1-9793-4105-9ae5-478d0a234e1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:12.388008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edffd1-9793-4105-9ae5-478d0a234e1a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:12.398603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 436/886] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:26:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:12.407806 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ea33d-0319-4269-8d29-10f1216096d0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:12.616050 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ea33d-0319-4269-8d29-10f1216096d0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:12.682103 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e89a95-ff58-46a2-b2b4-8e7b7b4ca2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:12.683192 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e89a95-ff58-46a2-b2b4-8e7b7b4ca2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:12.683695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e89a95-ff58-46a2-b2b4-8e7b7b4ca2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:12.684288 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e89a95-ff58-46a2-b2b4-8e7b7b4ca2a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:12.685771 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e89a95-ff58-46a2-b2b4-8e7b7b4ca2a9 None admin] RBAC: Adding query filter params (interface=public, service_id=c27129c10d214737b4f9d39375704c87, region_id=RegionOne) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:12.686498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e89a95-ff58-46a2-b2b4-8e7b7b4ca2a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:12.688913 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e89a95-ff58-46a2-b2b4-8e7b7b4ca2a9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:12.703774 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 451/887] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 18 08:26:12 2022] GET /identity/v3/endpoints?service_id=c27129c10d214737b4f9d39375704c87&interface=public®ion_id=RegionOne => generated 194 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:12.709391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edffd1-9793-4105-9ae5-478d0a234e1a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:12.767411 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:12.868847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edffd1-9793-4105-9ae5-478d0a234e1a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:12.936075 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052163b-dc8b-4726-9762-7429bf3a2a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:12.937264 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052163b-dc8b-4726-9762-7429bf3a2a9a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:12.937766 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052163b-dc8b-4726-9762-7429bf3a2a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:12.938574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052163b-dc8b-4726-9762-7429bf3a2a9a None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:12.939970 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052163b-dc8b-4726-9762-7429bf3a2a9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:12.943195 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052163b-dc8b-4726-9762-7429bf3a2a9a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:12.955904 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 437/888] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 18 08:26:12 2022] GET /identity/v3/services => generated 1268 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:26:14.084538 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792f50e-6001-4dc5-9829-79bf916cae07 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:14.085086 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792f50e-6001-4dc5-9829-79bf916cae07 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:14.085533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792f50e-6001-4dc5-9829-79bf916cae07 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:14.088676 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 452/889] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:14.094056 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab04563-2937-4584-8bf2-b7ad2653a5c1 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:14.094538 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab04563-2937-4584-8bf2-b7ad2653a5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:14.094538 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab04563-2937-4584-8bf2-b7ad2653a5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:14.114245 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-7ab04563-2937-4584-8bf2-b7ad2653a5c1 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:14.115598 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7ab04563-2937-4584-8bf2-b7ad2653a5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:14.217978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 438/890] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:14.262121 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792f50e-6001-4dc5-9829-79bf916cae07 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:14.369179 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:14.452546 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792f50e-6001-4dc5-9829-79bf916cae07 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:14.494849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8cc26-0e68-45ce-ba0a-d483a6f40bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:14.495547 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8cc26-0e68-45ce-ba0a-d483a6f40bc3 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:14.495871 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8cc26-0e68-45ce-ba0a-d483a6f40bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:14.496153 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8cc26-0e68-45ce-ba0a-d483a6f40bc3 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:14.496923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8cc26-0e68-45ce-ba0a-d483a6f40bc3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:14.498345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8cc26-0e68-45ce-ba0a-d483a6f40bc3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:14.578338 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 453/891] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 18 08:26:14 2022] POST /identity/v3/services => generated 241 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:26:14.703486 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5b6b7-0e40-4e93-ac4e-1d05e55c427b None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:14.704029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5b6b7-0e40-4e93-ac4e-1d05e55c427b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:14.704502 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5b6b7-0e40-4e93-ac4e-1d05e55c427b None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:14.708267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 439/892] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:14.714891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6149432-cd47-4331-bce7-df4b6cf513bb None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:14.715567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6149432-cd47-4331-bce7-df4b6cf513bb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:14.716086 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6149432-cd47-4331-bce7-df4b6cf513bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:14.735922 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a6149432-cd47-4331-bce7-df4b6cf513bb None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:14.737153 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a6149432-cd47-4331-bce7-df4b6cf513bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:14.825480 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 454/893] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:14.849514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5b6b7-0e40-4e93-ac4e-1d05e55c427b None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:14.899140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:15.015121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5b6b7-0e40-4e93-ac4e-1d05e55c427b None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:15.055233 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3843aa-6ba5-41ed-8be8-a32d0ffc92f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:15.056109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3843aa-6ba5-41ed-8be8-a32d0ffc92f2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:15.056395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3843aa-6ba5-41ed-8be8-a32d0ffc92f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:15.056671 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3843aa-6ba5-41ed-8be8-a32d0ffc92f2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:15.057345 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3843aa-6ba5-41ed-8be8-a32d0ffc92f2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:15.058744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3843aa-6ba5-41ed-8be8-a32d0ffc92f2 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:15.072497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-ef3843aa-6ba5-41ed-8be8-a32d0ffc92f2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 08:26:15.072497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:15.072497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:26:15.072497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:26:15.072497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:26:15.072497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:26:15.072497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:26:15.072497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:15.072497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 08:26:15.072497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:15.072497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:15.072497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:15.072497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:15.072497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:15.072497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:15.072497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:15.073605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:15.073605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:15.073605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:15.073605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:15.073605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:15.073605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:15.073605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:15.073605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:26:15.073605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:26:15.073605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:26:15.073605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:26:15.073605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:15.073605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:15.073605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:15.073605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:15.073605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:15.074521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:15.074521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:15.074521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:15.074521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:15.074521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:15.074521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:15.074521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:15.074521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:15.074521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:15.074521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:15.074521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:15.074521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:26:15.074521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:15.074521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 08:26:15.074521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:15.074521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 440/894] 10.222.0.57 () {56 vars in 1132 bytes} [Mon Jul 18 08:26:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:15.082016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6149432-cd47-4331-bce7-df4b6cf513bb None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:15.154839 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:15.253691 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6149432-cd47-4331-bce7-df4b6cf513bb None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:15.292118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d0ff6-d804-4716-91d5-7993f68cca48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:15.293552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d0ff6-d804-4716-91d5-7993f68cca48 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:15.294060 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d0ff6-d804-4716-91d5-7993f68cca48 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:15.294540 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d0ff6-d804-4716-91d5-7993f68cca48 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:15.295457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d0ff6-d804-4716-91d5-7993f68cca48 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:15.296096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d0ff6-d804-4716-91d5-7993f68cca48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:15.297779 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d0ff6-d804-4716-91d5-7993f68cca48 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:15.304397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 455/895] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 18 08:26:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:15.310337 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3843aa-6ba5-41ed-8be8-a32d0ffc92f2 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:15.387573 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd9c90-1d3b-4388-a157-43b4ff388895 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:15.388121 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd9c90-1d3b-4388-a157-43b4ff388895 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:15.388613 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd9c90-1d3b-4388-a157-43b4ff388895 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:15.391279 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 456/896] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:26:15.400468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98bfdb9-73ed-48fb-ace3-62d58d7055e1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:15.401388 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98bfdb9-73ed-48fb-ace3-62d58d7055e1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:15.402140 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98bfdb9-73ed-48fb-ace3-62d58d7055e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:15.429273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:15.431054 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a98bfdb9-73ed-48fb-ace3-62d58d7055e1 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:15.433571 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a98bfdb9-73ed-48fb-ace3-62d58d7055e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:15.453742 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3843aa-6ba5-41ed-8be8-a32d0ffc92f2 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:15.525409 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759b8e2-fda9-4a97-9551-017f72392aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:15.527294 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759b8e2-fda9-4a97-9551-017f72392aba None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:15.528184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759b8e2-fda9-4a97-9551-017f72392aba None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:15.528962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759b8e2-fda9-4a97-9551-017f72392aba None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:15.530531 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759b8e2-fda9-4a97-9551-017f72392aba None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:15.531330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759b8e2-fda9-4a97-9551-017f72392aba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:15.534368 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759b8e2-fda9-4a97-9551-017f72392aba None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:15.546339 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 441/897] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 18 08:26:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:15.555759 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759b8e2-fda9-4a97-9551-017f72392aba None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:15.579571 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 457/898] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:15.622452 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98bfdb9-73ed-48fb-ace3-62d58d7055e1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:15.677355 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759b8e2-fda9-4a97-9551-017f72392aba None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:15.683775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:15.714489 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f2a59-44f1-4fae-b886-14ffaee307d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:15.715154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f2a59-44f1-4fae-b886-14ffaee307d2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:15.715467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f2a59-44f1-4fae-b886-14ffaee307d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:15.715752 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f2a59-44f1-4fae-b886-14ffaee307d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:15.716522 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f2a59-44f1-4fae-b886-14ffaee307d2 None admin] RBAC: Adding query filter params (interface=public, service_id=46226e3b1f42431ea0be9f724a1de22c, region_id=RegionOne) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:15.716829 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f2a59-44f1-4fae-b886-14ffaee307d2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:15.718622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f2a59-44f1-4fae-b886-14ffaee307d2 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:15.727208 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 442/899] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 18 08:26:15 2022] GET /identity/v3/endpoints?service_id=46226e3b1f42431ea0be9f724a1de22c&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:15.728623 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccdd8e-22c9-4c5f-b402-6cb74911bdac None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:15.728913 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccdd8e-22c9-4c5f-b402-6cb74911bdac None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:15.729194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccdd8e-22c9-4c5f-b402-6cb74911bdac None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:15.730780 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 443/900] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:15.735504 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ccdd8e-22c9-4c5f-b402-6cb74911bdac None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:15.760735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:15.793378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98bfdb9-73ed-48fb-ace3-62d58d7055e1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:15.832141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe8573-24b1-4321-b18b-659a66ee5488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:15.836423 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe8573-24b1-4321-b18b-659a66ee5488 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:15.836423 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe8573-24b1-4321-b18b-659a66ee5488 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:15.836423 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe8573-24b1-4321-b18b-659a66ee5488 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:15.853930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fe8573-24b1-4321-b18b-659a66ee5488 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:26:15.871454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-24fe8573-24b1-4321-b18b-659a66ee5488 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 08:26:15.871454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:15.871454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:15.871454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:15.871454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:15.871454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:15.871454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:15.871454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:15.871454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:15.871454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:15.871454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:15.871454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:15.871454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:26:15.871454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:26:15.871454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:26:15.871454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:26:15.872636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:26:15.872636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:26:15.872636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:26:15.872636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:26:15.872636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:15.872636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:15.872636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:15.872636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:15.872636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:15.872636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:15.872636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:15.872636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:15.872636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:15.872636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:15.872636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:15.872636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:15.873499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:15.873499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:15.873499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:15.873499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:15.873499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:26:15.873499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:26:15.873499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:26:15.873499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:26:15.873499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:26:15.873499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:26:15.873499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 08:26:15.873499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:15.874203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 458/901] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Jul 18 08:26:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:15.876296 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0955ac3-a197-4dee-8944-b5adde06948a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:15.876797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0955ac3-a197-4dee-8944-b5adde06948a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:15.877279 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0955ac3-a197-4dee-8944-b5adde06948a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:15.891391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ccdd8e-22c9-4c5f-b402-6cb74911bdac None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:15.905512 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f0955ac3-a197-4dee-8944-b5adde06948a None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:15.907392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f0955ac3-a197-4dee-8944-b5adde06948a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:15.929604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8d849-9a85-49e8-bde8-216dcf6918fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:15.930706 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8d849-9a85-49e8-bde8-216dcf6918fa None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:15.931213 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8d849-9a85-49e8-bde8-216dcf6918fa None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:15.931785 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8d849-9a85-49e8-bde8-216dcf6918fa None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:15.932680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8d849-9a85-49e8-bde8-216dcf6918fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:15.934272 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8d849-9a85-49e8-bde8-216dcf6918fa None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:15.941181 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 444/902] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 18 08:26:15 2022] GET /identity/v3/services => generated 1497 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:26:15.943182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8d849-9a85-49e8-bde8-216dcf6918fa None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:16.031091 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 459/903] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:16.055532 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0955ac3-a197-4dee-8944-b5adde06948a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:16.091471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:16.104161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:16.128712 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8d849-9a85-49e8-bde8-216dcf6918fa None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:16.178017 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24830957-493e-4720-bca6-2cecedca8474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:16.178707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24830957-493e-4720-bca6-2cecedca8474 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:16.178995 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24830957-493e-4720-bca6-2cecedca8474 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:16.179277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24830957-493e-4720-bca6-2cecedca8474 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:16.180141 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24830957-493e-4720-bca6-2cecedca8474 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:16.180456 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24830957-493e-4720-bca6-2cecedca8474 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:16.182033 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24830957-493e-4720-bca6-2cecedca8474 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:16.192786 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 445/904] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 18 08:26:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:16.200781 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24830957-493e-4720-bca6-2cecedca8474 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:16.268588 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0955ac3-a197-4dee-8944-b5adde06948a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:16.311966 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57abfc10-ad84-46ea-8f84-5574d6a634ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:16.312780 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57abfc10-ad84-46ea-8f84-5574d6a634ca None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:16.313062 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57abfc10-ad84-46ea-8f84-5574d6a634ca None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:16.313326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57abfc10-ad84-46ea-8f84-5574d6a634ca None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:16.314121 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57abfc10-ad84-46ea-8f84-5574d6a634ca None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:16.315372 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57abfc10-ad84-46ea-8f84-5574d6a634ca None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:16.328611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-57abfc10-ad84-46ea-8f84-5574d6a634ca None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 08:26:16.328611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:16.328611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:26:16.328611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:26:16.328611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:26:16.328611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:26:16.328611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:26:16.328611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:16.328611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 08:26:16.328611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:16.328611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:16.328611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:16.328611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:16.328611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:16.328611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:16.328611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:16.328611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:16.330850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:16.330850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:16.330850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:16.330850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:16.330850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:16.330850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:16.330850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:26:16.330850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:26:16.330850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:26:16.330850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:26:16.330850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:16.330850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:16.330850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:16.330850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:16.330850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:16.330850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:16.332942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:16.332942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:16.332942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:16.332942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:16.332942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:16.332942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:16.332942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:16.332942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:16.332942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:16.332942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:16.332942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:26:16.332942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:16.332942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 08:26:16.332942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:16.332942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 460/905] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:26:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:16.336022 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57abfc10-ad84-46ea-8f84-5574d6a634ca None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:16.396782 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24830957-493e-4720-bca6-2cecedca8474 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:16.448040 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c27304-3116-4c78-81ce-0fff118acb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:16.448871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c27304-3116-4c78-81ce-0fff118acb3e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:16.449287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c27304-3116-4c78-81ce-0fff118acb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:16.449740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c27304-3116-4c78-81ce-0fff118acb3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:16.466528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c27304-3116-4c78-81ce-0fff118acb3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:16.469108 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c27304-3116-4c78-81ce-0fff118acb3e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:16.497134 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:16.508110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-36c27304-3116-4c78-81ce-0fff118acb3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:16.508110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:16.508110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:16.508110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:16.508110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:16.508110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:16.508110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:16.508110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:16.508110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:16.508110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:16.508110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:16.508110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:16.508110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:16.508110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:16.508110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:16.510078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:16.510078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:16.510078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:16.510078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:16.510078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:16.510078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:16.510078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:16.510078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:16.510078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:16.510078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:16.510078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:16.510078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:16.510078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:16.510078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:16.510078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:16.510078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:16.510078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:16.517893 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:16.517893 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:16.517893 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:16.517893 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:16.517893 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 446/906] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:16.518276 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:16.518364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c27304-3116-4c78-81ce-0fff118acb3e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:16.552569 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57abfc10-ad84-46ea-8f84-5574d6a634ca None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:16.599163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c009ef-a762-4f32-87ef-46055de09eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:16.599897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c009ef-a762-4f32-87ef-46055de09eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:16.600182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c009ef-a762-4f32-87ef-46055de09eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:16.600457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c009ef-a762-4f32-87ef-46055de09eb4 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:16.601185 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c009ef-a762-4f32-87ef-46055de09eb4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:16.601492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c009ef-a762-4f32-87ef-46055de09eb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:16.603016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c009ef-a762-4f32-87ef-46055de09eb4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:16.609527 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 461/907] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:26:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:16.616169 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c009ef-a762-4f32-87ef-46055de09eb4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:16.702610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c27304-3116-4c78-81ce-0fff118acb3e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:16.750608 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8b432-f1ba-47ad-ae23-e20c5c7abb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:16.751288 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8b432-f1ba-47ad-ae23-e20c5c7abb79 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:16.751604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8b432-f1ba-47ad-ae23-e20c5c7abb79 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:16.752046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8b432-f1ba-47ad-ae23-e20c5c7abb79 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:16.752653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8b432-f1ba-47ad-ae23-e20c5c7abb79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:16.752965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8b432-f1ba-47ad-ae23-e20c5c7abb79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:16.755214 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8b432-f1ba-47ad-ae23-e20c5c7abb79 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:16.770610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 447/908] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:16.780420 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8b432-f1ba-47ad-ae23-e20c5c7abb79 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:16.886324 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c009ef-a762-4f32-87ef-46055de09eb4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:16.916543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:16.954622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc174b-2a7c-4a07-9a60-54b11792930f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:16.955596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc174b-2a7c-4a07-9a60-54b11792930f None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:16.955937 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc174b-2a7c-4a07-9a60-54b11792930f None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:16.956357 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc174b-2a7c-4a07-9a60-54b11792930f None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:16.957145 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc174b-2a7c-4a07-9a60-54b11792930f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:16.960385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc174b-2a7c-4a07-9a60-54b11792930f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:16.963376 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc174b-2a7c-4a07-9a60-54b11792930f None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:16.974496 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 462/909] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Jul 18 08:26:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:16.979885 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc174b-2a7c-4a07-9a60-54b11792930f None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:16.994012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8b432-f1ba-47ad-ae23-e20c5c7abb79 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:17.016539 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:17.044223 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc5e1e-02b9-4429-a9c4-92a1898de287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:17.045572 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc5e1e-02b9-4429-a9c4-92a1898de287 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:17.046182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc5e1e-02b9-4429-a9c4-92a1898de287 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:17.046854 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc5e1e-02b9-4429-a9c4-92a1898de287 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:17.058746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc5e1e-02b9-4429-a9c4-92a1898de287 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:17.060886 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc5e1e-02b9-4429-a9c4-92a1898de287 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:17.079082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-adbc5e1e-02b9-4429-a9c4-92a1898de287 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 08:26:17.079082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:17.079082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:17.079082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:17.079082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:17.079082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:17.079082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:17.079082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:17.079082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:17.079082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:17.079082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:17.079082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:17.079082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:26:17.079082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:26:17.079082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:26:17.079082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:26:17.080330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:17.080330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:17.080330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:26:17.080330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:17.080330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:26:17.080330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:17.080330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:17.080330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:17.080330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:17.080330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:17.080330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:17.080330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:17.080330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:17.080330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:17.080330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:17.080330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:17.080330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:17.080330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:17.081316 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:17.081316 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:17.081316 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:26:17.081316 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:26:17.081316 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:26:17.081316 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:26:17.081316 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:26:17.081316 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:26:17.081316 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 08:26:17.081316 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:17.089419 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 448/910] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 18 08:26:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:17.090832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc5e1e-02b9-4429-a9c4-92a1898de287 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:17.135022 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc174b-2a7c-4a07-9a60-54b11792930f None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:17.183295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6e88a-179e-417e-8e4d-2c82e68f21a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:17.184022 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6e88a-179e-417e-8e4d-2c82e68f21a7 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:17.184380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6e88a-179e-417e-8e4d-2c82e68f21a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:17.185322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6e88a-179e-417e-8e4d-2c82e68f21a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:17.186201 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6e88a-179e-417e-8e4d-2c82e68f21a7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:17.187995 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6e88a-179e-417e-8e4d-2c82e68f21a7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:17.221999 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc5e1e-02b9-4429-a9c4-92a1898de287 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:17.243020 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:17.267853 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6576434-bc79-4e43-99d9-d1ae89956f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:17.268556 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6576434-bc79-4e43-99d9-d1ae89956f2c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:17.268877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6576434-bc79-4e43-99d9-d1ae89956f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:17.269161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6576434-bc79-4e43-99d9-d1ae89956f2c None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:17.270563 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6576434-bc79-4e43-99d9-d1ae89956f2c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:17.270924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6576434-bc79-4e43-99d9-d1ae89956f2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:17.272558 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6576434-bc79-4e43-99d9-d1ae89956f2c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:17.287967 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 463/911] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 18 08:26:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:26:17.305828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 449/912] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Jul 18 08:26:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:17.312918 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6e88a-179e-417e-8e4d-2c82e68f21a7 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:17.452562 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:17.508496 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6e88a-179e-417e-8e4d-2c82e68f21a7 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:17.561002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f150ac-cca5-464d-89f1-79f6082b88af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:17.561967 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f150ac-cca5-464d-89f1-79f6082b88af None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:17.562316 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f150ac-cca5-464d-89f1-79f6082b88af None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:17.562749 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f150ac-cca5-464d-89f1-79f6082b88af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:17.660950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f150ac-cca5-464d-89f1-79f6082b88af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:17.663064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f150ac-cca5-464d-89f1-79f6082b88af None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:17.671150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daede29-124d-461f-a84f-95784a23365c None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:17.671717 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daede29-124d-461f-a84f-95784a23365c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:17.671907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daede29-124d-461f-a84f-95784a23365c None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:17.673439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 450/913] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:26:17.679649 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f44365-2a69-4886-8d29-79d9f22d07b1 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:17.680026 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f44365-2a69-4886-8d29-79d9f22d07b1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:17.680259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f44365-2a69-4886-8d29-79d9f22d07b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:17.700330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a0f44365-2a69-4886-8d29-79d9f22d07b1 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:17.701603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a0f44365-2a69-4886-8d29-79d9f22d07b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:17.769670 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:17.796041 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 451/914] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:17.814542 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f44365-2a69-4886-8d29-79d9f22d07b1 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:17.845941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 464/915] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:26:17 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 08:26:17.853918 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f150ac-cca5-464d-89f1-79f6082b88af None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:17.940921 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f44365-2a69-4886-8d29-79d9f22d07b1 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:17.943574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:18.000340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c4e82-5292-466c-a681-8da48327a4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:18.001969 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c4e82-5292-466c-a681-8da48327a4be None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:18.002712 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c4e82-5292-466c-a681-8da48327a4be None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:18.003455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c4e82-5292-466c-a681-8da48327a4be None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:18.005637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c4e82-5292-466c-a681-8da48327a4be None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:18.008605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c4e82-5292-466c-a681-8da48327a4be None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:18.029863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f150ac-cca5-464d-89f1-79f6082b88af None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:18.032617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-503c4e82-5292-466c-a681-8da48327a4be None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 08:26:18.032617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:18.032617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:26:18.032617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:26:18.032617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:26:18.032617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:26:18.032617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:26:18.032617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:18.032617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 08:26:18.032617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:18.032617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:18.032617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:18.032617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:18.032617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:18.032617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:18.032617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:18.032617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:18.034023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:18.034023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:18.034023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:18.034023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:18.034023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:18.034023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:18.034023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:26:18.034023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:26:18.034023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:26:18.034023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:26:18.034023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:18.034023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:18.034023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:18.034023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:18.034023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:18.034023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:18.035566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:18.035566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:18.035566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:18.035566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:18.035566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:18.035566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:18.035566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:18.035566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:18.035566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:18.035566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:18.035566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:26:18.035566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:18.035566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 08:26:18.035566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:18.039521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 452/916] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 18 08:26:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:18.048746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c4e82-5292-466c-a681-8da48327a4be None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:18.104060 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6da99-1125-4466-b462-d43eba37cd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:18.106131 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6da99-1125-4466-b462-d43eba37cd47 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:18.106935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6da99-1125-4466-b462-d43eba37cd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:18.107787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6da99-1125-4466-b462-d43eba37cd47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:18.121328 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6da99-1125-4466-b462-d43eba37cd47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:18.123338 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6da99-1125-4466-b462-d43eba37cd47 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:18.143180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-a2e6da99-1125-4466-b462-d43eba37cd47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:18.143180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:18.143180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:18.143180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:18.143180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:18.143180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:18.143180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:18.143180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:18.143180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:18.143180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:18.143180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:18.143180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:18.143180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:18.143180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:18.143180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:18.144182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:18.144182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:18.144182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:18.144182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:18.144182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:18.144182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:18.144182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:18.144182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:18.144182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:18.144182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:18.144182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:18.144182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:18.144182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:18.144182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:18.144182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:18.144182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:18.144182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:18.145618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:18.145618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:18.145618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:18.145618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:18.152681 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 465/917] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:18.154023 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6da99-1125-4466-b462-d43eba37cd47 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:18.176901 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:18.253821 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c4e82-5292-466c-a681-8da48327a4be None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:18.282930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6da99-1125-4466-b462-d43eba37cd47 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:18.304703 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b40650-60d1-4a2f-90f6-64eab8687ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:18.305658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b40650-60d1-4a2f-90f6-64eab8687ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:18.305947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b40650-60d1-4a2f-90f6-64eab8687ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:18.306229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b40650-60d1-4a2f-90f6-64eab8687ec6 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:18.306956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b40650-60d1-4a2f-90f6-64eab8687ec6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:18.307262 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b40650-60d1-4a2f-90f6-64eab8687ec6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:18.308732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b40650-60d1-4a2f-90f6-64eab8687ec6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:18.319133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 453/918] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 18 08:26:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:18.320482 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af07b6-11fc-43e0-bfff-a98a02b7c19f None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:18.320778 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af07b6-11fc-43e0-bfff-a98a02b7c19f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:18.321064 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af07b6-11fc-43e0-bfff-a98a02b7c19f None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:18.322429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 454/919] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:18.323148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af07b6-11fc-43e0-bfff-a98a02b7c19f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:18.328879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee446e-3468-48bf-9260-ec80cc1cbbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:18.329702 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee446e-3468-48bf-9260-ec80cc1cbbd5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:18.330044 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee446e-3468-48bf-9260-ec80cc1cbbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:18.330427 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee446e-3468-48bf-9260-ec80cc1cbbd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:18.331881 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee446e-3468-48bf-9260-ec80cc1cbbd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:18.333054 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee446e-3468-48bf-9260-ec80cc1cbbd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:18.335203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee446e-3468-48bf-9260-ec80cc1cbbd5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:18.356533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 466/920] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:18.358776 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac4fbf-952a-46fe-83ea-667cd1b6a9fe None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:18.359279 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac4fbf-952a-46fe-83ea-667cd1b6a9fe None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:18.359828 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac4fbf-952a-46fe-83ea-667cd1b6a9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:18.368097 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:18.383513 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-53ac4fbf-952a-46fe-83ea-667cd1b6a9fe None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:18.385031 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-53ac4fbf-952a-46fe-83ea-667cd1b6a9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:18.463499 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af07b6-11fc-43e0-bfff-a98a02b7c19f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:18.505243 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb4806-c697-45a1-9c6b-61b20a8b59c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:18.505949 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb4806-c697-45a1-9c6b-61b20a8b59c8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:18.506229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb4806-c697-45a1-9c6b-61b20a8b59c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:18.506501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb4806-c697-45a1-9c6b-61b20a8b59c8 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:18.507293 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb4806-c697-45a1-9c6b-61b20a8b59c8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:18.507596 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb4806-c697-45a1-9c6b-61b20a8b59c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:18.509073 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb4806-c697-45a1-9c6b-61b20a8b59c8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:18.511712 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 467/921] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:18.511712 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac4fbf-952a-46fe-83ea-667cd1b6a9fe None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:18.527729 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 455/922] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 18 08:26:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:18.528988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb4806-c697-45a1-9c6b-61b20a8b59c8 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:18.531544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:18.667018 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:18.673112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac4fbf-952a-46fe-83ea-667cd1b6a9fe None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:18.673847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb4806-c697-45a1-9c6b-61b20a8b59c8 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:18.719590 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47430388-a6b2-4d32-87fd-560bd091bd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:18.720391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47430388-a6b2-4d32-87fd-560bd091bd10 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:18.720747 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47430388-a6b2-4d32-87fd-560bd091bd10 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:18.720950 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47430388-a6b2-4d32-87fd-560bd091bd10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:18.721756 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47430388-a6b2-4d32-87fd-560bd091bd10 None admin] RBAC: Adding query filter params (interface=public, service_id=da877297515048abb21ff6a6ff7f1b23, region_id=RegionOne) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:18.722068 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47430388-a6b2-4d32-87fd-560bd091bd10 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:18.723988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47430388-a6b2-4d32-87fd-560bd091bd10 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:18.727205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6caff8-5306-4517-b35f-db046b9d98db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:18.728088 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6caff8-5306-4517-b35f-db046b9d98db None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:18.728364 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6caff8-5306-4517-b35f-db046b9d98db None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:18.728753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6caff8-5306-4517-b35f-db046b9d98db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:18.731095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 456/923] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 18 08:26:18 2022] GET /identity/v3/endpoints?service_id=da877297515048abb21ff6a6ff7f1b23&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:18.731990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47430388-a6b2-4d32-87fd-560bd091bd10 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:18.746853 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6caff8-5306-4517-b35f-db046b9d98db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:18.749460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6caff8-5306-4517-b35f-db046b9d98db None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:18.776705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-df6caff8-5306-4517-b35f-db046b9d98db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:18.776705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:18.776705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:18.776705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:18.776705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:18.776705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:18.776705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:18.776705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:18.776705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:18.776705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:18.776705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:18.776705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:18.776705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:26:18.776705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:26:18.776705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:26:18.777798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:26:18.777798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:18.777798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:18.777798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:18.777798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:18.777798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:18.777798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:18.777798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:18.777798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:18.777798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:18.777798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:18.777798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:18.777798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:18.777798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:18.777798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:18.777798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:18.777798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:18.779215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:26:18.779215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:26:18.779215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:26:18.779215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:26:18.779215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:26:18.779215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:26:18.779215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:18.779215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:18.779215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 468/924] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:26:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:18.780438 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6caff8-5306-4517-b35f-db046b9d98db None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:18.881410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47430388-a6b2-4d32-87fd-560bd091bd10 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:18.902992 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:18.934017 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4366f-0e52-4ac6-9eee-371b93111376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:18.934724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4366f-0e52-4ac6-9eee-371b93111376 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:18.935010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4366f-0e52-4ac6-9eee-371b93111376 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:18.935284 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4366f-0e52-4ac6-9eee-371b93111376 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:18.935996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4366f-0e52-4ac6-9eee-371b93111376 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:18.937390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4366f-0e52-4ac6-9eee-371b93111376 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:18.947121 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6caff8-5306-4517-b35f-db046b9d98db None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:18.953811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-ccf4366f-0e52-4ac6-9eee-371b93111376 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 08:26:18.953811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:18.953811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:26:18.953811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:26:18.953811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:26:18.953811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:26:18.953811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:26:18.953811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:18.953811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 08:26:18.953811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:18.953811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:18.953811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:18.953811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:18.953811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:18.953811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:18.953811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:18.955744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:18.955744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:18.955744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:18.955744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:18.955744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:18.955744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:18.955744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:18.955744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:26:18.955744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:26:18.955744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:26:18.955744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:26:18.955744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:18.955744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:18.955744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:18.955744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:18.955744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:18.957956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:18.957956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:18.957956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:18.957956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:18.957956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:18.957956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:18.957956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:18.957956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:18.957956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:18.957956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:18.957956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:18.957956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:26:18.957956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:18.957956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 08:26:18.957956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:18.957956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 457/925] 10.222.0.57 () {56 vars in 1132 bytes} [Mon Jul 18 08:26:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:18.961236 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4366f-0e52-4ac6-9eee-371b93111376 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:19.010740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:19.021056 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fdec4a-7e3c-406a-8710-05a73c6bae74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:19.021753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fdec4a-7e3c-406a-8710-05a73c6bae74 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:19.022071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fdec4a-7e3c-406a-8710-05a73c6bae74 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:19.022387 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fdec4a-7e3c-406a-8710-05a73c6bae74 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:19.023064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fdec4a-7e3c-406a-8710-05a73c6bae74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:19.024536 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fdec4a-7e3c-406a-8710-05a73c6bae74 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:19.033105 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 469/926] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 18 08:26:18 2022] GET /identity/v3/services => generated 1497 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:26:19.034619 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fdec4a-7e3c-406a-8710-05a73c6bae74 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:19.184509 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4366f-0e52-4ac6-9eee-371b93111376 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:19.253093 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fdec4a-7e3c-406a-8710-05a73c6bae74 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:19.264359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e447c-a42c-4c70-84b8-76bf8399a9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:19.265489 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e447c-a42c-4c70-84b8-76bf8399a9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:19.265916 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e447c-a42c-4c70-84b8-76bf8399a9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:19.266383 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e447c-a42c-4c70-84b8-76bf8399a9f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:19.268044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e447c-a42c-4c70-84b8-76bf8399a9f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:19.268593 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e447c-a42c-4c70-84b8-76bf8399a9f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:19.273617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e447c-a42c-4c70-84b8-76bf8399a9f0 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:19.299902 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 458/927] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:26:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:19.310886 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e447c-a42c-4c70-84b8-76bf8399a9f0 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:19.343831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:19.354303 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0acbe9-bc4d-42c4-9c60-5e40e9796374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:19.356780 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0acbe9-bc4d-42c4-9c60-5e40e9796374 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:19.356780 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0acbe9-bc4d-42c4-9c60-5e40e9796374 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:19.357263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0acbe9-bc4d-42c4-9c60-5e40e9796374 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:19.358878 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0acbe9-bc4d-42c4-9c60-5e40e9796374 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:19.359567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0acbe9-bc4d-42c4-9c60-5e40e9796374 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:19.362785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0acbe9-bc4d-42c4-9c60-5e40e9796374 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:19.374316 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 470/928] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 18 08:26:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:19.385166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0acbe9-bc4d-42c4-9c60-5e40e9796374 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:19.475190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:19.511925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e447c-a42c-4c70-84b8-76bf8399a9f0 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:19.566861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28171a22-1b50-4c9d-9766-6f9ea6e96503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:19.567686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28171a22-1b50-4c9d-9766-6f9ea6e96503 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:19.567982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28171a22-1b50-4c9d-9766-6f9ea6e96503 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:19.568259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28171a22-1b50-4c9d-9766-6f9ea6e96503 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:19.569114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28171a22-1b50-4c9d-9766-6f9ea6e96503 None admin] RBAC: Adding query filter params (role.id=9fd88609ddd942d5a9b4bc684e5e49b2, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=2b03f8b8c51e472bbd4cf0afc9c6b632) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:19.569427 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28171a22-1b50-4c9d-9766-6f9ea6e96503 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:19.570907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28171a22-1b50-4c9d-9766-6f9ea6e96503 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:19.579801 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 459/929] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:26:19 2022] GET /identity/v3/role_assignments?user.id=2b03f8b8c51e472bbd4cf0afc9c6b632&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=9fd88609ddd942d5a9b4bc684e5e49b2 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:19.595943 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0acbe9-bc4d-42c4-9c60-5e40e9796374 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:19.666549 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95214584-a47e-4bec-8580-4da5f4e5f974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:19.667740 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95214584-a47e-4bec-8580-4da5f4e5f974 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:19.668255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95214584-a47e-4bec-8580-4da5f4e5f974 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:19.668733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95214584-a47e-4bec-8580-4da5f4e5f974 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:19.670169 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95214584-a47e-4bec-8580-4da5f4e5f974 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:19.670719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95214584-a47e-4bec-8580-4da5f4e5f974 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:19.673257 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95214584-a47e-4bec-8580-4da5f4e5f974 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:19.684164 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 471/930] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 18 08:26:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:19.694715 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28171a22-1b50-4c9d-9766-6f9ea6e96503 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:19.875407 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28171a22-1b50-4c9d-9766-6f9ea6e96503 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:19.941944 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8524c44-ddb5-41e5-a52b-073a59938741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:19.943582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8524c44-ddb5-41e5-a52b-073a59938741 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:19.944391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8524c44-ddb5-41e5-a52b-073a59938741 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:19.945190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8524c44-ddb5-41e5-a52b-073a59938741 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:19.946930 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8524c44-ddb5-41e5-a52b-073a59938741 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:19.949853 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8524c44-ddb5-41e5-a52b-073a59938741 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:20.034870 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73340eb3-a2f2-43b8-98b6-ce404383d7a9 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:20.035905 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73340eb3-a2f2-43b8-98b6-ce404383d7a9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:20.036062 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73340eb3-a2f2-43b8-98b6-ce404383d7a9 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:20.037781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 472/931] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:20.043844 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513ecf8-8f7c-40b8-a078-f0d6a8ddd735 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:20.044258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513ecf8-8f7c-40b8-a078-f0d6a8ddd735 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:20.044449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513ecf8-8f7c-40b8-a078-f0d6a8ddd735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:20.050529 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 460/932] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 18 08:26:19 2022] POST /identity/v3/endpoints => generated 347 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:26:20.064388 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a513ecf8-8f7c-40b8-a078-f0d6a8ddd735 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:20.065737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a513ecf8-8f7c-40b8-a078-f0d6a8ddd735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:20.147113 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:20.171819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 473/933] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:20.194394 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8524c44-ddb5-41e5-a52b-073a59938741 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:20.199026 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:20.362644 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8524c44-ddb5-41e5-a52b-073a59938741 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:20.434274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a986a-100c-4be9-9798-6979cd432dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:20.435961 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a986a-100c-4be9-9798-6979cd432dab None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:20.436732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a986a-100c-4be9-9798-6979cd432dab None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:20.437593 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a986a-100c-4be9-9798-6979cd432dab None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:20.439144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a986a-100c-4be9-9798-6979cd432dab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:20.442210 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a986a-100c-4be9-9798-6979cd432dab None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:20.445135 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: WARNING keystone.server.flask.common [None req-957a986a-100c-4be9-9798-6979cd432dab None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 18 08:26:20.497298 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.sql.core [None req-957a986a-100c-4be9-9798-6979cd432dab None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 08:26:20.497298 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 08:26:20.497298 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [parameters: {'id': '0e851d54e9074e87ac59d4b23a34582e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 08:26:20.497298 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80338) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 08:26:20.516012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-957a986a-100c-4be9-9798-6979cd432dab None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0e851d54e9074e87ac59d4b23a34582e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0e851d54e9074e87ac59d4b23a34582e. Jul 18 08:26:20.516012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:20.516012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 08:26:20.516012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 08:26:20.516012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 08:26:20.516012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 08:26:20.516012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 08:26:20.516012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 08:26:20.516012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 08:26:20.516012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application conn.query(q) Jul 18 08:26:20.516012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 08:26:20.516012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 08:26:20.516012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 08:26:20.517822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application result.read() Jul 18 08:26:20.517822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 08:26:20.517822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 08:26:20.517822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 08:26:20.517822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 08:26:20.517822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 08:26:20.517822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 08:26:20.517822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 08:26:20.517822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 08:26:20.517822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 08:26:20.517822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:20.517822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 08:26:20.517822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:20.517822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:20.517822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 08:26:20.517822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 08:26:20.517822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 18 08:26:20.517822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 18 08:26:20.520840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 08:26:20.520840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application next(self.gen) Jul 18 08:26:20.520840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 08:26:20.520840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application yield resource Jul 18 08:26:20.520840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 08:26:20.520840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application next(self.gen) Jul 18 08:26:20.520840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 08:26:20.520840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 08:26:20.520840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 08:26:20.520840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 08:26:20.520840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 08:26:20.520840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise self.value Jul 18 08:26:20.520840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 08:26:20.520840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 08:26:20.520840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 08:26:20.520840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application session.commit() Jul 18 08:26:20.520840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 08:26:20.524737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 08:26:20.524737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 08:26:20.524737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 08:26:20.524737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 08:26:20.524737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self.session.flush() Jul 18 08:26:20.524737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 08:26:20.524737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 08:26:20.524737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 08:26:20.524737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 08:26:20.524737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 08:26:20.524737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application compat.raise_( Jul 18 08:26:20.524737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 08:26:20.524737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception Jul 18 08:26:20.524737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 08:26:20.524737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 08:26:20.524737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 08:26:20.524737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 08:26:20.531689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 08:26:20.531689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 08:26:20.531689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 08:26:20.531689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 08:26:20.531689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 08:26:20.531689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 08:26:20.531689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 08:26:20.531689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 08:26:20.531689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 08:26:20.531689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 08:26:20.531689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 08:26:20.531689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 08:26:20.531689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 08:26:20.531689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 08:26:20.531689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 08:26:20.535841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 08:26:20.535841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 08:26:20.535841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception Jul 18 08:26:20.535841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 08:26:20.535841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 08:26:20.535841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 08:26:20.535841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 08:26:20.535841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 08:26:20.535841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 08:26:20.535841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 08:26:20.535841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application conn.query(q) Jul 18 08:26:20.535841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 08:26:20.535841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 08:26:20.535841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 08:26:20.535841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application result.read() Jul 18 08:26:20.535841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 08:26:20.535841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 08:26:20.540996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 08:26:20.540996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 08:26:20.540996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 08:26:20.540996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 08:26:20.540996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 08:26:20.540996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 08:26:20.540996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 08:26:20.540996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 08:26:20.540996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [parameters: {'id': '0e851d54e9074e87ac59d4b23a34582e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 08:26:20.540996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 08:26:20.540996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:20.540996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:20.540996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:20.540996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:20.540996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 18 08:26:20.547196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 18 08:26:20.547196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 08:26:20.547196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 08:26:20.547196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 18 08:26:20.547196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:20.547196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:20.547196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:20.547196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:20.547196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:20.547196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:20.547196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:20.547196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:20.547196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:20.547196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:20.547196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:20.547196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:20.547196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:20.550846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:20.550846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 18 08:26:20.550846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 18 08:26:20.550846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:20.550846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:20.550846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 18 08:26:20.550846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 18 08:26:20.550846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0e851d54e9074e87ac59d4b23a34582e. Jul 18 08:26:20.550846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:20.550846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 461/934] 10.222.0.57 () {60 vars in 1140 bytes} [Mon Jul 18 08:26:20 2022] POST /identity/v3/projects => generated 262 bytes in 328 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:20.555549 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513ecf8-8f7c-40b8-a078-f0d6a8ddd735 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:20.777920 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513ecf8-8f7c-40b8-a078-f0d6a8ddd735 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:20.850049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73a2df-340c-4d54-b31b-a7d7f10c50b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:20.850876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73a2df-340c-4d54-b31b-a7d7f10c50b4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:20.851165 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73a2df-340c-4d54-b31b-a7d7f10c50b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:20.851428 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73a2df-340c-4d54-b31b-a7d7f10c50b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:20.868596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73a2df-340c-4d54-b31b-a7d7f10c50b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:20.871755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73a2df-340c-4d54-b31b-a7d7f10c50b4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:20.901787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-9c73a2df-340c-4d54-b31b-a7d7f10c50b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:20.901787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:20.901787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:20.901787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:20.901787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:20.901787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:20.901787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:20.901787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:20.901787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:20.901787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:20.901787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:20.901787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:20.901787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:26:20.901787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:26:20.901787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:26:20.903605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:26:20.903605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:20.903605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:20.903605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:20.903605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:20.903605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:20.903605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:20.903605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:20.903605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:20.903605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:20.903605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:20.903605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:20.903605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:20.903605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:20.903605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:20.903605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:20.903605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:20.906737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:26:20.906737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:26:20.906737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:26:20.906737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:26:20.906737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:26:20.906737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:26:20.906737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:20.906737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:20.906737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 474/935] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:26:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:20.915953 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a986a-100c-4be9-9798-6979cd432dab None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:21.005012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:21.025669 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a986a-100c-4be9-9798-6979cd432dab None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:21.063567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed5935d-3bd8-4884-b9ed-c807292dade7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:21.064539 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5935d-3bd8-4884-b9ed-c807292dade7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:21.064970 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5935d-3bd8-4884-b9ed-c807292dade7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:21.065405 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5935d-3bd8-4884-b9ed-c807292dade7 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:21.067058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5935d-3bd8-4884-b9ed-c807292dade7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:21.067575 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5935d-3bd8-4884-b9ed-c807292dade7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:21.069661 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5935d-3bd8-4884-b9ed-c807292dade7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:21.092690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 462/936] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 18 08:26:20 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:21.843373 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1378be0-914a-41c4-8061-ae9008b1e242 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:21.844191 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1378be0-914a-41c4-8061-ae9008b1e242 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:21.844624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1378be0-914a-41c4-8061-ae9008b1e242 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:21.846203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 475/937] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:21.854407 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc901357-fac7-4632-8f7e-398993bb25b4 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:21.854705 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc901357-fac7-4632-8f7e-398993bb25b4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:21.854990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc901357-fac7-4632-8f7e-398993bb25b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:21.873392 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-bc901357-fac7-4632-8f7e-398993bb25b4 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:21.874964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bc901357-fac7-4632-8f7e-398993bb25b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:21.951830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 463/938] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:21.974941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1378be0-914a-41c4-8061-ae9008b1e242 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:22.032071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:22.128494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1378be0-914a-41c4-8061-ae9008b1e242 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:22.129931 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3c1c4-170e-44b1-a3c7-614500e50a29 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:22.130304 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3c1c4-170e-44b1-a3c7-614500e50a29 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:22.130659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3c1c4-170e-44b1-a3c7-614500e50a29 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:22.132700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 464/939] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:22.138307 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c420adc-eb8f-4cb1-a367-dc66b1b3c2a0 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:22.138771 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c420adc-eb8f-4cb1-a367-dc66b1b3c2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:22.139130 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c420adc-eb8f-4cb1-a367-dc66b1b3c2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:22.157414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0c420adc-eb8f-4cb1-a367-dc66b1b3c2a0 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:22.158738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0c420adc-eb8f-4cb1-a367-dc66b1b3c2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:22.178078 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055aa801-5148-4800-b92d-fd08503bff74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:22.179558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055aa801-5148-4800-b92d-fd08503bff74 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:22.180098 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055aa801-5148-4800-b92d-fd08503bff74 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:22.180435 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055aa801-5148-4800-b92d-fd08503bff74 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:22.197214 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055aa801-5148-4800-b92d-fd08503bff74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:26:22.206321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-055aa801-5148-4800-b92d-fd08503bff74 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 08:26:22.206321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:22.206321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:22.206321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:22.206321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:22.206321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:22.206321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:22.206321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:22.206321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:22.206321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:22.206321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:22.206321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:22.206321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:26:22.206321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:26:22.206321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:26:22.206321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:26:22.208069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:26:22.208069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:26:22.208069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:26:22.208069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:26:22.208069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:22.208069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:22.208069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:22.208069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:22.208069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:22.208069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:22.208069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:22.208069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:22.208069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:22.208069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:22.208069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:22.208069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:22.208938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:22.208938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:22.208938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:22.208938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:22.208938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:26:22.208938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:26:22.208938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:26:22.208938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:26:22.208938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:26:22.208938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:26:22.208938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 08:26:22.208938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:22.211457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 476/940] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Jul 18 08:26:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:22.217337 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055aa801-5148-4800-b92d-fd08503bff74 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:22.223110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:22.280171 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 465/941] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:22.304612 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c420adc-eb8f-4cb1-a367-dc66b1b3c2a0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:22.347087 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055aa801-5148-4800-b92d-fd08503bff74 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:22.356936 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:22.391492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d52520-84b5-42bb-84db-04714d7062ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:22.392157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d52520-84b5-42bb-84db-04714d7062ee None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:22.392446 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d52520-84b5-42bb-84db-04714d7062ee None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:22.392722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d52520-84b5-42bb-84db-04714d7062ee None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:22.393496 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d52520-84b5-42bb-84db-04714d7062ee None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:22.393835 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d52520-84b5-42bb-84db-04714d7062ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:22.395265 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d52520-84b5-42bb-84db-04714d7062ee None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:22.405210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 477/942] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 18 08:26:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:22.411717 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d52520-84b5-42bb-84db-04714d7062ee None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:22.454499 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c420adc-eb8f-4cb1-a367-dc66b1b3c2a0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:22.500136 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550dfa3-d00d-4221-9e1c-7a8f4c5eedef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:22.501363 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550dfa3-d00d-4221-9e1c-7a8f4c5eedef None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:22.501872 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550dfa3-d00d-4221-9e1c-7a8f4c5eedef None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:22.502363 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550dfa3-d00d-4221-9e1c-7a8f4c5eedef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:22.503895 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9550dfa3-d00d-4221-9e1c-7a8f4c5eedef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:22.506212 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9550dfa3-d00d-4221-9e1c-7a8f4c5eedef None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:22.527583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-9550dfa3-d00d-4221-9e1c-7a8f4c5eedef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 08:26:22.527583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:22.527583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:26:22.527583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:26:22.527583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:26:22.527583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:26:22.527583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:26:22.527583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:22.527583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 08:26:22.527583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:22.527583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:22.527583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:22.527583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:22.527583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:22.527583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:22.527583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:22.527583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:22.528689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:22.528689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:22.528689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:22.528689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:22.528689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:22.528689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:22.528689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:26:22.528689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:26:22.528689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:26:22.528689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:26:22.528689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:22.528689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:22.528689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:22.528689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:22.528689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:22.528689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:22.529543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:22.529543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:22.529543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:22.529543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:22.529543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:22.529543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:22.529543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:22.529543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:22.529543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:22.529543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:22.529543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:26:22.529543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:22.529543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 08:26:22.529543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:22.532645 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 466/943] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 18 08:26:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:22.538992 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550dfa3-d00d-4221-9e1c-7a8f4c5eedef None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:22.594568 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d52520-84b5-42bb-84db-04714d7062ee None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:22.608045 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:22.642093 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d3ffc-6f33-4e4c-b4a4-92a460d379c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:22.643240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d3ffc-6f33-4e4c-b4a4-92a460d379c7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:22.643763 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d3ffc-6f33-4e4c-b4a4-92a460d379c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:22.644252 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d3ffc-6f33-4e4c-b4a4-92a460d379c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:22.657348 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d3ffc-6f33-4e4c-b4a4-92a460d379c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:22.659349 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d3ffc-6f33-4e4c-b4a4-92a460d379c7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:22.679653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-af3d3ffc-6f33-4e4c-b4a4-92a460d379c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:22.679653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:22.679653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:22.679653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:22.679653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:22.679653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:22.679653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:22.679653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:22.679653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:22.679653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:22.679653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:22.679653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:22.679653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:22.679653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:22.679653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:22.680692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:22.680692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:22.680692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:22.680692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:22.680692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:22.680692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:22.680692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:22.680692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:22.680692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:22.680692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:22.680692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:22.680692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:22.680692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:22.680692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:22.680692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:22.680692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:22.680692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:22.681597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:22.681597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:22.681597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:22.681597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:22.684078 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 478/944] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:22.690367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d3ffc-6f33-4e4c-b4a4-92a460d379c7 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:22.707462 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550dfa3-d00d-4221-9e1c-7a8f4c5eedef None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:22.736594 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:22.770160 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4256b1c-aa97-496f-8a1d-3900a216d6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:22.771872 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4256b1c-aa97-496f-8a1d-3900a216d6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:22.772770 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4256b1c-aa97-496f-8a1d-3900a216d6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:22.773471 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4256b1c-aa97-496f-8a1d-3900a216d6f4 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:22.774929 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4256b1c-aa97-496f-8a1d-3900a216d6f4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:22.775721 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4256b1c-aa97-496f-8a1d-3900a216d6f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:22.778423 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4256b1c-aa97-496f-8a1d-3900a216d6f4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:22.788853 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 467/945] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 18 08:26:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:22.794500 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4256b1c-aa97-496f-8a1d-3900a216d6f4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:22.935937 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d3ffc-6f33-4e4c-b4a4-92a460d379c7 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:23.005556 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb023be-9d21-4441-94d4-bb49f1c83930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:23.007501 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb023be-9d21-4441-94d4-bb49f1c83930 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:23.008973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb023be-9d21-4441-94d4-bb49f1c83930 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:23.009713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb023be-9d21-4441-94d4-bb49f1c83930 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:23.011618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb023be-9d21-4441-94d4-bb49f1c83930 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:23.012453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb023be-9d21-4441-94d4-bb49f1c83930 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:23.016377 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb023be-9d21-4441-94d4-bb49f1c83930 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:23.048038 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4256b1c-aa97-496f-8a1d-3900a216d6f4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:23.055776 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 479/946] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:23.060955 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb023be-9d21-4441-94d4-bb49f1c83930 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:23.068543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:23.122096 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2563e81-fd2c-4fbe-a0cd-74a2e87aebb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:23.123920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2563e81-fd2c-4fbe-a0cd-74a2e87aebb4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:23.124830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2563e81-fd2c-4fbe-a0cd-74a2e87aebb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:23.125314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2563e81-fd2c-4fbe-a0cd-74a2e87aebb4 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:23.126663 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2563e81-fd2c-4fbe-a0cd-74a2e87aebb4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:23.127194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2563e81-fd2c-4fbe-a0cd-74a2e87aebb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:23.130475 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2563e81-fd2c-4fbe-a0cd-74a2e87aebb4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:23.145590 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 468/947] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 18 08:26:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:23.145590 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce58e4-cc16-44bf-bcae-fdceaf5068b2 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:23.145590 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce58e4-cc16-44bf-bcae-fdceaf5068b2 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:23.145590 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce58e4-cc16-44bf-bcae-fdceaf5068b2 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:23.151739 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 469/948] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:23.151739 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce58e4-cc16-44bf-bcae-fdceaf5068b2 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:23.220586 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:23.255088 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb023be-9d21-4441-94d4-bb49f1c83930 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:23.306601 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce58e4-cc16-44bf-bcae-fdceaf5068b2 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:23.315115 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba845708-914e-4e12-b043-35d064ffe652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:23.317370 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba845708-914e-4e12-b043-35d064ffe652 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:23.318110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba845708-914e-4e12-b043-35d064ffe652 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:23.318604 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba845708-914e-4e12-b043-35d064ffe652 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:23.332681 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba845708-914e-4e12-b043-35d064ffe652 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:23.335467 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba845708-914e-4e12-b043-35d064ffe652 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:23.348385 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146f880-90bd-4939-8f30-f8dbd929b9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:23.349064 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146f880-90bd-4939-8f30-f8dbd929b9d0 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:23.349486 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146f880-90bd-4939-8f30-f8dbd929b9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:23.349761 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146f880-90bd-4939-8f30-f8dbd929b9d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:23.350524 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146f880-90bd-4939-8f30-f8dbd929b9d0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:23.351963 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146f880-90bd-4939-8f30-f8dbd929b9d0 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:23.367590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-ba845708-914e-4e12-b043-35d064ffe652 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 08:26:23.367590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:23.367590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:23.367590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:23.367590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:23.367590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:23.367590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:23.367590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:23.367590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:23.367590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:23.367590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:23.367590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:23.367590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:26:23.367590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:26:23.367590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:26:23.367590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:26:23.369368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:23.369368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:23.369368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:26:23.369368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:23.369368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:26:23.369368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:23.369368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:23.369368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:23.369368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:23.369368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:23.369368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:23.369368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:23.369368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:23.369368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:23.369368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:23.369368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:23.369368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:23.369368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:23.371484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:23.371484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:23.371484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:26:23.371484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:26:23.371484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:26:23.371484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:26:23.371484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:26:23.371484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:26:23.371484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 08:26:23.371484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:23.371484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 480/949] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 18 08:26:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:23.371484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58643968-77c4-478c-a400-8f03cfec870a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:23.372942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58643968-77c4-478c-a400-8f03cfec870a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:23.372942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58643968-77c4-478c-a400-8f03cfec870a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:23.402386 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-58643968-77c4-478c-a400-8f03cfec870a None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:23.404575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-58643968-77c4-478c-a400-8f03cfec870a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:23.451763 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 470/950] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 18 08:26:23 2022] POST /identity/v3/endpoints => generated 349 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:26:23.452686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146f880-90bd-4939-8f30-f8dbd929b9d0 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:23.494705 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:23.525596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 481/951] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:23.527222 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39800f-793b-44d4-b664-26aeb513bed2 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:23.527637 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39800f-793b-44d4-b664-26aeb513bed2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:23.528052 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39800f-793b-44d4-b664-26aeb513bed2 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:23.530045 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 482/952] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:23.536470 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692c946-51ed-4256-84f4-0c2c7fd9c4c1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:23.536828 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692c946-51ed-4256-84f4-0c2c7fd9c4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:23.537197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692c946-51ed-4256-84f4-0c2c7fd9c4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:23.557924 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-4692c946-51ed-4256-84f4-0c2c7fd9c4c1 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:23.559502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4692c946-51ed-4256-84f4-0c2c7fd9c4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:23.627508 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146f880-90bd-4939-8f30-f8dbd929b9d0 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:23.654242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:23.676166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 483/953] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:23.676166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692c946-51ed-4256-84f4-0c2c7fd9c4c1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:23.686832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcf395-f237-4121-ae64-ad3b2851e727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:23.689789 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcf395-f237-4121-ae64-ad3b2851e727 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:23.690390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcf395-f237-4121-ae64-ad3b2851e727 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:23.690894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcf395-f237-4121-ae64-ad3b2851e727 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:23.693187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcf395-f237-4121-ae64-ad3b2851e727 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:23.693766 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcf395-f237-4121-ae64-ad3b2851e727 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:23.696821 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcf395-f237-4121-ae64-ad3b2851e727 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:23.723931 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 471/954] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Jul 18 08:26:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:23.725275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcf395-f237-4121-ae64-ad3b2851e727 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:23.827724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692c946-51ed-4256-84f4-0c2c7fd9c4c1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:23.855175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:23.870832 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ef943-c6e4-4c23-8364-bd3a702e1245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:23.871714 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ef943-c6e4-4c23-8364-bd3a702e1245 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:23.871998 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ef943-c6e4-4c23-8364-bd3a702e1245 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:23.872270 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ef943-c6e4-4c23-8364-bd3a702e1245 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:23.872944 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ef943-c6e4-4c23-8364-bd3a702e1245 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:23.874345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ef943-c6e4-4c23-8364-bd3a702e1245 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:23.875438 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcf395-f237-4121-ae64-ad3b2851e727 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:23.889753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-581ef943-c6e4-4c23-8364-bd3a702e1245 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 08:26:23.889753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:23.889753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:26:23.889753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:26:23.889753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:26:23.889753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:26:23.889753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:26:23.889753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:23.889753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 08:26:23.889753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:23.889753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:23.889753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:23.889753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:23.889753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:23.889753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:23.889753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:23.889753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:23.891972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:23.891972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:23.891972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:23.891972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:23.891972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:23.891972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:23.891972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:26:23.891972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:26:23.891972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:26:23.891972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:26:23.891972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:23.891972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:23.891972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:23.891972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:23.891972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:23.891972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:23.893849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:23.893849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:23.893849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:23.893849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:23.893849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:23.893849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:23.893849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:23.893849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:23.893849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:23.893849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:23.893849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:26:23.893849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:23.893849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 08:26:23.893849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:23.893849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 484/955] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:26:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:23.897079 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ef943-c6e4-4c23-8364-bd3a702e1245 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:23.929648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9c4f4-cbb1-4943-99eb-a3d048966b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:23.930346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9c4f4-cbb1-4943-99eb-a3d048966b1e None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:23.930635 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9c4f4-cbb1-4943-99eb-a3d048966b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:23.930917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9c4f4-cbb1-4943-99eb-a3d048966b1e None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:23.931919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9c4f4-cbb1-4943-99eb-a3d048966b1e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:23.933714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9c4f4-cbb1-4943-99eb-a3d048966b1e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:23.971389 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:23.989137 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.sql.core [None req-7da9c4f4-cbb1-4943-99eb-a3d048966b1e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 08:26:23.989137 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 08:26:23.989137 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [parameters: {'user_id': '07a3a86ed8a1418e9599fab26e8d9b05', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 08:26:23.989137 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80338) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 08:26:23.998580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-7da9c4f4-cbb1-4943-99eb-a3d048966b1e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 08:26:23.998580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:23.998580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 08:26:23.998580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 08:26:23.998580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 08:26:23.998580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 08:26:23.998580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 08:26:23.998580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 08:26:23.998580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 08:26:23.998580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application conn.query(q) Jul 18 08:26:23.998580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 08:26:23.998580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 08:26:23.998580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 08:26:23.998580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application result.read() Jul 18 08:26:23.998580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 08:26:24.000434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 08:26:24.000434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 08:26:24.000434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 08:26:24.000434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 08:26:24.000434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 08:26:24.000434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 08:26:24.000434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 08:26:24.000434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 08:26:24.000434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:24.000434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 08:26:24.000434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:24.000434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:24.000434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 08:26:24.000434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 08:26:24.000434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 18 08:26:24.000434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 18 08:26:24.000434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 08:26:24.000434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application next(self.gen) Jul 18 08:26:24.002432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 08:26:24.002432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application yield resource Jul 18 08:26:24.002432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 08:26:24.002432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application next(self.gen) Jul 18 08:26:24.002432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 08:26:24.002432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 08:26:24.002432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 08:26:24.002432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 08:26:24.002432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 08:26:24.002432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise self.value Jul 18 08:26:24.002432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 08:26:24.002432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 08:26:24.002432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 08:26:24.002432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application session.commit() Jul 18 08:26:24.002432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 08:26:24.002432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 08:26:24.002432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 08:26:24.007201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 08:26:24.007201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 08:26:24.007201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self.session.flush() Jul 18 08:26:24.007201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 08:26:24.007201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 08:26:24.007201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 08:26:24.007201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 08:26:24.007201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 08:26:24.007201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application compat.raise_( Jul 18 08:26:24.007201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 08:26:24.007201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception Jul 18 08:26:24.007201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 08:26:24.007201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 08:26:24.007201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 08:26:24.007201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 08:26:24.007201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 08:26:24.007201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 08:26:24.009514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 08:26:24.009514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 08:26:24.009514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 18 08:26:24.009514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 18 08:26:24.009514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 08:26:24.009514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 08:26:24.009514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 08:26:24.009514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 08:26:24.009514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 08:26:24.009514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 08:26:24.009514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 08:26:24.009514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 08:26:24.009514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 08:26:24.009514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 08:26:24.009514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 08:26:24.009514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception Jul 18 08:26:24.011102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 08:26:24.011102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 08:26:24.011102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 08:26:24.011102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 08:26:24.011102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 08:26:24.011102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 08:26:24.011102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 08:26:24.011102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application conn.query(q) Jul 18 08:26:24.011102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 08:26:24.011102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 08:26:24.011102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 08:26:24.011102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application result.read() Jul 18 08:26:24.011102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 08:26:24.011102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 08:26:24.011102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 08:26:24.011102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 08:26:24.015183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 08:26:24.015183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 08:26:24.015183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 08:26:24.015183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 08:26:24.015183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 08:26:24.015183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 08:26:24.015183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [parameters: {'user_id': '07a3a86ed8a1418e9599fab26e8d9b05', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 08:26:24.015183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 08:26:24.015183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:24.015183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:24.015183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:24.015183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:24.015183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:24.015183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:24.015183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:24.018629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:24.018629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:24.018629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:24.018629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:24.018629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:24.018629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:24.018629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:24.018629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 18 08:26:24.018629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 18 08:26:24.018629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:24.018629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:24.018629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:26:24.018629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:24.018629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:26:24.018629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:24.018629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 18 08:26:24.018629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 18 08:26:24.022914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 18 08:26:24.022914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 18 08:26:24.022914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 08:26:24.022914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 08:26:24.022914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 08:26:24.022914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:24.022914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 472/956] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Jul 18 08:26:23 2022] POST /identity/v3/users => generated 160 bytes in 276 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:24.022914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9c4f4-cbb1-4943-99eb-a3d048966b1e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:24.036321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ef943-c6e4-4c23-8364-bd3a702e1245 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:24.078623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccac584-aa0f-469b-a77e-2ff7896cd8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:24.079437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccac584-aa0f-469b-a77e-2ff7896cd8ae None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:24.079824 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccac584-aa0f-469b-a77e-2ff7896cd8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:24.080162 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccac584-aa0f-469b-a77e-2ff7896cd8ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:24.094156 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccac584-aa0f-469b-a77e-2ff7896cd8ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:24.095731 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccac584-aa0f-469b-a77e-2ff7896cd8ae None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:24.118151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-bccac584-aa0f-469b-a77e-2ff7896cd8ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:24.118151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:24.118151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:24.118151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:24.118151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:24.118151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:24.118151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:24.118151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:24.118151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:24.118151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:24.118151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:24.118151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:24.118151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:24.118151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:24.118151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:24.119135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:24.119135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:24.119135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:24.119135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:24.119135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:24.119135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:24.119135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:24.119135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:24.119135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:24.119135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:24.119135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:24.119135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:24.119135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:24.119135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:24.119135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:24.119135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:24.119135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:24.120075 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:24.120075 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:24.120075 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:24.120075 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:24.120075 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 485/957] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:24.120825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccac584-aa0f-469b-a77e-2ff7896cd8ae None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:24.152930 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9c4f4-cbb1-4943-99eb-a3d048966b1e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:24.205986 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ae7a1-c3b7-4def-b3f3-2d457706a174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:24.207490 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ae7a1-c3b7-4def-b3f3-2d457706a174 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:24.207836 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ae7a1-c3b7-4def-b3f3-2d457706a174 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:24.208110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ae7a1-c3b7-4def-b3f3-2d457706a174 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:24.208866 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ae7a1-c3b7-4def-b3f3-2d457706a174 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:24.209177 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ae7a1-c3b7-4def-b3f3-2d457706a174 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:24.210813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ae7a1-c3b7-4def-b3f3-2d457706a174 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:24.227869 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 473/958] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 18 08:26:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:24.227869 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ae7a1-c3b7-4def-b3f3-2d457706a174 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:24.242704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:24.267699 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccac584-aa0f-469b-a77e-2ff7896cd8ae None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:24.293882 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:24.331306 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265f610-273f-451e-93c8-75b2cf23366e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:24.333186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265f610-273f-451e-93c8-75b2cf23366e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:24.333688 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265f610-273f-451e-93c8-75b2cf23366e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:24.334183 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265f610-273f-451e-93c8-75b2cf23366e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:24.344313 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ae7a1-c3b7-4def-b3f3-2d457706a174 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:24.350884 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265f610-273f-451e-93c8-75b2cf23366e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:24.353642 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265f610-273f-451e-93c8-75b2cf23366e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:24.371155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-9265f610-273f-451e-93c8-75b2cf23366e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 08:26:24.371155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:24.371155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:24.371155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:24.371155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:24.371155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:24.371155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:24.371155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:24.371155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:24.371155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:24.371155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:24.371155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:24.371155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:26:24.371155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:26:24.371155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:26:24.371155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:26:24.372266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:24.372266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:24.372266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:26:24.372266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:24.372266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:26:24.372266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:24.372266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:24.372266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:24.372266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:24.372266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:24.372266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:24.372266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:24.372266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:24.372266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:24.372266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:24.372266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:24.372266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:24.372266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:24.373203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:24.373203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:24.373203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:26:24.373203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:26:24.373203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:26:24.373203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:26:24.373203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:26:24.373203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:26:24.373203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 08:26:24.373203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:24.377360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 486/959] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:26:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:24.377360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265f610-273f-451e-93c8-75b2cf23366e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:24.392796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8752a6-54b2-442e-b8d3-9db726ffc1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:24.393474 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8752a6-54b2-442e-b8d3-9db726ffc1b6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:24.393761 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8752a6-54b2-442e-b8d3-9db726ffc1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:24.394036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8752a6-54b2-442e-b8d3-9db726ffc1b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:24.394816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8752a6-54b2-442e-b8d3-9db726ffc1b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:24.395130 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8752a6-54b2-442e-b8d3-9db726ffc1b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:24.396543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8752a6-54b2-442e-b8d3-9db726ffc1b6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:24.415242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 474/960] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:24.415242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8752a6-54b2-442e-b8d3-9db726ffc1b6 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:24.488737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:24.527107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8752a6-54b2-442e-b8d3-9db726ffc1b6 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:24.563561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265f610-273f-451e-93c8-75b2cf23366e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:24.566016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccde34-4425-47fe-b715-69c76dbb220f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:24.567502 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccde34-4425-47fe-b715-69c76dbb220f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:24.567917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccde34-4425-47fe-b715-69c76dbb220f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:24.568131 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccde34-4425-47fe-b715-69c76dbb220f None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:24.568920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ccde34-4425-47fe-b715-69c76dbb220f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:24.569239 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ccde34-4425-47fe-b715-69c76dbb220f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:24.570698 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ccde34-4425-47fe-b715-69c76dbb220f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:24.619792 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 475/961] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Jul 18 08:26:24 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:24.619792 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccde34-4425-47fe-b715-69c76dbb220f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:24.638674 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012bcfb-ad40-4466-b9c3-cb346890c51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:24.639888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012bcfb-ad40-4466-b9c3-cb346890c51d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:24.640491 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012bcfb-ad40-4466-b9c3-cb346890c51d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:24.641078 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012bcfb-ad40-4466-b9c3-cb346890c51d None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:24.642881 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012bcfb-ad40-4466-b9c3-cb346890c51d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:24.643528 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012bcfb-ad40-4466-b9c3-cb346890c51d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:24.647059 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012bcfb-ad40-4466-b9c3-cb346890c51d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:24.656863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 487/962] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 18 08:26:24 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:24.756427 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccde34-4425-47fe-b715-69c76dbb220f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:24.793500 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:24.807027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae337953-7b04-4d93-a277-029db6b2c74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:24.809569 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae337953-7b04-4d93-a277-029db6b2c74e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:24.809569 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae337953-7b04-4d93-a277-029db6b2c74e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:24.809569 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae337953-7b04-4d93-a277-029db6b2c74e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:24.823463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae337953-7b04-4d93-a277-029db6b2c74e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:24.826720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae337953-7b04-4d93-a277-029db6b2c74e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:24.849648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-ae337953-7b04-4d93-a277-029db6b2c74e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:24.849648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:24.849648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:24.849648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:24.849648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:24.849648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:24.849648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:24.849648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:24.849648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:24.849648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:24.849648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:24.849648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:24.849648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:26:24.849648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:26:24.849648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:26:24.851391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:26:24.851391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:24.851391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:24.851391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:24.851391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:24.851391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:24.851391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:24.851391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:24.851391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:24.851391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:24.851391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:24.851391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:24.851391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:24.851391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:24.851391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:24.851391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:24.851391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:24.853267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:26:24.853267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:26:24.853267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:26:24.853267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:26:24.853267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:26:24.853267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:26:24.853267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:24.853267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:24.853267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 476/963] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:26:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:24.859567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012bcfb-ad40-4466-b9c3-cb346890c51d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:24.917843 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:25.040521 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012bcfb-ad40-4466-b9c3-cb346890c51d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:25.089742 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d6922-672d-432a-bae2-96980cb33005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:25.090410 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d6922-672d-432a-bae2-96980cb33005 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:25.090694 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d6922-672d-432a-bae2-96980cb33005 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:25.090969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d6922-672d-432a-bae2-96980cb33005 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:25.091797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d6922-672d-432a-bae2-96980cb33005 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:25.092109 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d6922-672d-432a-bae2-96980cb33005 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:25.093593 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d6922-672d-432a-bae2-96980cb33005 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:25.110994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 488/964] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:26:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:25.112592 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae337953-7b04-4d93-a277-029db6b2c74e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:25.293534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae337953-7b04-4d93-a277-029db6b2c74e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:25.342127 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9738cbe-6ddb-4122-b0e2-9877bd7339e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:25.343235 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9738cbe-6ddb-4122-b0e2-9877bd7339e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:25.343648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9738cbe-6ddb-4122-b0e2-9877bd7339e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:25.344091 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9738cbe-6ddb-4122-b0e2-9877bd7339e5 None admin] PATH_INFO: `/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/2b03f8b8c51e472bbd4cf0afc9c6b632/roles/9fd88609ddd942d5a9b4bc684e5e49b2` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:25.346872 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9738cbe-6ddb-4122-b0e2-9877bd7339e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=35c846931fd443fbb83a6fbe0aeaa866, user_id=2b03f8b8c51e472bbd4cf0afc9c6b632, role_id=9fd88609ddd942d5a9b4bc684e5e49b2)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:25.348601 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9738cbe-6ddb-4122-b0e2-9877bd7339e5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:25.432205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 477/965] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:26:25 2022] PUT /identity/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/2b03f8b8c51e472bbd4cf0afc9c6b632/roles/9fd88609ddd942d5a9b4bc684e5e49b2 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:26:26.405900 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf77b27-b83c-4665-a16d-aa3048ba7145 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:26.406716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf77b27-b83c-4665-a16d-aa3048ba7145 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:26.406716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf77b27-b83c-4665-a16d-aa3048ba7145 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:26.408244 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 489/966] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:26:26.414506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed808eb-2726-476e-a5aa-558cd8617b58 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:26.415312 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed808eb-2726-476e-a5aa-558cd8617b58 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:26.415312 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed808eb-2726-476e-a5aa-558cd8617b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:26.433398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-9ed808eb-2726-476e-a5aa-558cd8617b58 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:26.434951 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9ed808eb-2726-476e-a5aa-558cd8617b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:26.496486 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:26.545290 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 478/967] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:26.586492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf77b27-b83c-4665-a16d-aa3048ba7145 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:26.784071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:26.797299 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf77b27-b83c-4665-a16d-aa3048ba7145 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:26.858615 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7492e-0d5b-49ce-bed7-283343e02f2f None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:26.859019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7492e-0d5b-49ce-bed7-283343e02f2f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:26.859207 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7492e-0d5b-49ce-bed7-283343e02f2f None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:26.860711 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 479/968] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:26:26.866915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a65b5-be1f-48ed-b213-5b43307b9503 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:26.867203 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a65b5-be1f-48ed-b213-5b43307b9503 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:26.867657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a65b5-be1f-48ed-b213-5b43307b9503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:26.867877 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec8767-ae91-45c1-8ee2-c575cf382c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:26.869058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec8767-ae91-45c1-8ee2-c575cf382c28 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:26.869525 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec8767-ae91-45c1-8ee2-c575cf382c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:26.869985 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec8767-ae91-45c1-8ee2-c575cf382c28 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:26.871145 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec8767-ae91-45c1-8ee2-c575cf382c28 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:26.873910 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec8767-ae91-45c1-8ee2-c575cf382c28 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:26.886982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-448a65b5-be1f-48ed-b213-5b43307b9503 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:26.888342 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-448a65b5-be1f-48ed-b213-5b43307b9503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:26.902412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-c4ec8767-ae91-45c1-8ee2-c575cf382c28 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 08:26:26.902412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:26.902412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:26:26.902412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:26:26.902412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:26:26.902412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:26:26.902412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:26:26.902412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:26.902412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 08:26:26.902412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:26.902412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:26.902412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:26.902412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:26.902412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:26.902412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:26.902412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:26.902412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:26.904220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:26.904220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:26.904220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:26.904220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:26.904220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:26.904220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:26.904220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:26:26.904220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:26:26.904220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:26:26.904220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:26:26.904220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:26.904220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:26.904220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:26.904220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:26.904220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:26.904220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:26.905896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:26.905896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:26.905896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:26.905896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:26.905896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:26.905896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:26.905896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:26.905896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:26.905896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:26.905896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:26.905896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:26:26.905896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:26.905896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 08:26:26.905896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:26.905896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 490/969] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 18 08:26:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:26.910991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec8767-ae91-45c1-8ee2-c575cf382c28 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:26.995479 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 480/970] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:27.020191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a65b5-be1f-48ed-b213-5b43307b9503 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:27.061278 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec8767-ae91-45c1-8ee2-c575cf382c28 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:27.103294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be58f1a-bea9-44a4-b14f-189b3b738d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:27.104200 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be58f1a-bea9-44a4-b14f-189b3b738d83 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:27.104550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be58f1a-bea9-44a4-b14f-189b3b738d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:27.104829 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be58f1a-bea9-44a4-b14f-189b3b738d83 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:27.105704 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be58f1a-bea9-44a4-b14f-189b3b738d83 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:27.106015 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be58f1a-bea9-44a4-b14f-189b3b738d83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:27.107533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be58f1a-bea9-44a4-b14f-189b3b738d83 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:27.119719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 491/971] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 18 08:26:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:27.120921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be58f1a-bea9-44a4-b14f-189b3b738d83 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:27.151239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:27.168381 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a65b5-be1f-48ed-b213-5b43307b9503 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:27.179980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:27.208801 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3457099-9a82-4ec8-99b4-23dfcb7867d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:27.209797 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3457099-9a82-4ec8-99b4-23dfcb7867d4 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:27.210232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3457099-9a82-4ec8-99b4-23dfcb7867d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:27.210719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3457099-9a82-4ec8-99b4-23dfcb7867d4 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:27.211857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3457099-9a82-4ec8-99b4-23dfcb7867d4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:27.213804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3457099-9a82-4ec8-99b4-23dfcb7867d4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:27.254461 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be58f1a-bea9-44a4-b14f-189b3b738d83 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:27.287890 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 481/972] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 18 08:26:27 2022] POST /identity/v3/services => generated 242 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:26:27.328816 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905a818-0603-49cd-928f-0f8aecb627cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:27.329976 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905a818-0603-49cd-928f-0f8aecb627cb None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:27.331300 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905a818-0603-49cd-928f-0f8aecb627cb None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:27.331300 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905a818-0603-49cd-928f-0f8aecb627cb None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:27.333235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905a818-0603-49cd-928f-0f8aecb627cb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:27.335709 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905a818-0603-49cd-928f-0f8aecb627cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:27.338014 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905a818-0603-49cd-928f-0f8aecb627cb None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:27.350812 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 492/973] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 18 08:26:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:27.356562 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3457099-9a82-4ec8-99b4-23dfcb7867d4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:27.493647 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3457099-9a82-4ec8-99b4-23dfcb7867d4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:27.541119 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb0705-b02e-4687-98c7-43caea55423a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:27.542155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb0705-b02e-4687-98c7-43caea55423a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:27.548949 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb0705-b02e-4687-98c7-43caea55423a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:27.548949 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb0705-b02e-4687-98c7-43caea55423a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:27.548949 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb0705-b02e-4687-98c7-43caea55423a None admin] RBAC: Adding query filter params (interface=admin, service_id=da877297515048abb21ff6a6ff7f1b23, region_id=RegionOne) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:27.548949 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb0705-b02e-4687-98c7-43caea55423a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:27.548949 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb0705-b02e-4687-98c7-43caea55423a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:27.553749 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 482/974] 10.222.0.57 () {56 vars in 1249 bytes} [Mon Jul 18 08:26:27 2022] GET /identity/v3/endpoints?service_id=da877297515048abb21ff6a6ff7f1b23&interface=admin®ion_id=RegionOne => generated 193 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:27.559994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905a818-0603-49cd-928f-0f8aecb627cb None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:27.673329 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:27.691507 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905a818-0603-49cd-928f-0f8aecb627cb None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:27.733394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5dd6f2-fbbf-4476-89e3-577560494230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:27.734067 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dd6f2-fbbf-4476-89e3-577560494230 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:27.734400 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dd6f2-fbbf-4476-89e3-577560494230 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:27.734705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dd6f2-fbbf-4476-89e3-577560494230 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:27.735373 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5dd6f2-fbbf-4476-89e3-577560494230 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:27.736827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5dd6f2-fbbf-4476-89e3-577560494230 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:27.751728 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 493/975] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 18 08:26:27 2022] GET /identity/v3/services => generated 1727 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:26:28.686005 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122ecd8-df0a-4e9b-b7fe-21812595d295 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:28.686636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122ecd8-df0a-4e9b-b7fe-21812595d295 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:28.686995 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122ecd8-df0a-4e9b-b7fe-21812595d295 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:28.689994 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 483/976] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:28.699108 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71b2fe-a945-41f7-a2ff-459263bf1f84 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:28.699902 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71b2fe-a945-41f7-a2ff-459263bf1f84 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:28.700175 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71b2fe-a945-41f7-a2ff-459263bf1f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:28.729746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7d71b2fe-a945-41f7-a2ff-459263bf1f84 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:28.731812 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7d71b2fe-a945-41f7-a2ff-459263bf1f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:28.857689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 494/977] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:28.897183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122ecd8-df0a-4e9b-b7fe-21812595d295 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:28.959443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:29.099816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122ecd8-df0a-4e9b-b7fe-21812595d295 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:29.162128 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d75cb7-6341-4aff-b0c8-41e28a940d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:29.163518 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d75cb7-6341-4aff-b0c8-41e28a940d92 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:29.164056 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d75cb7-6341-4aff-b0c8-41e28a940d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:29.164534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d75cb7-6341-4aff-b0c8-41e28a940d92 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:29.186785 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d75cb7-6341-4aff-b0c8-41e28a940d92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:26:29.204030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-81d75cb7-6341-4aff-b0c8-41e28a940d92 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 08:26:29.204030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:29.204030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:29.204030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:29.204030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:29.204030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:29.204030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:29.204030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:29.204030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:29.204030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:29.204030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:29.204030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:29.204030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:26:29.204030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:26:29.204030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:26:29.204030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:26:29.205912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:26:29.205912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:26:29.205912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:26:29.205912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:26:29.205912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:29.205912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:29.205912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:29.205912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:29.205912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:29.205912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:29.205912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:29.205912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:29.205912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:29.205912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:29.205912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:29.205912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:29.213636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:29.213636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:29.213636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:29.213636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:29.213636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:26:29.213636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:26:29.213636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:26:29.213636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:26:29.213636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:26:29.213636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:26:29.213636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 08:26:29.213636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:29.213636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 484/978] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Jul 18 08:26:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:29.219970 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71b2fe-a945-41f7-a2ff-459263bf1f84 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:29.381590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:29.422678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71b2fe-a945-41f7-a2ff-459263bf1f84 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:29.492568 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a8398-642b-4a39-a4b8-e506d2c63a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:29.493743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a8398-642b-4a39-a4b8-e506d2c63a5e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:29.494245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a8398-642b-4a39-a4b8-e506d2c63a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:29.494746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a8398-642b-4a39-a4b8-e506d2c63a5e None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:29.496185 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0a8398-642b-4a39-a4b8-e506d2c63a5e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:29.496733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0a8398-642b-4a39-a4b8-e506d2c63a5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:29.499247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0a8398-642b-4a39-a4b8-e506d2c63a5e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:29.523711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 495/979] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 18 08:26:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:29.527283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d75cb7-6341-4aff-b0c8-41e28a940d92 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:29.535771 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393f59e-175d-4195-b84f-4825ea87db4b None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:29.536263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393f59e-175d-4195-b84f-4825ea87db4b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:29.536676 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393f59e-175d-4195-b84f-4825ea87db4b None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:29.538299 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 496/980] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:26:29.544611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceadb0b-d5e6-417a-9c01-f7c68229a99c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:29.545151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceadb0b-d5e6-417a-9c01-f7c68229a99c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:29.545523 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceadb0b-d5e6-417a-9c01-f7c68229a99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:29.565286 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3ceadb0b-d5e6-417a-9c01-f7c68229a99c None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:29.567454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3ceadb0b-d5e6-417a-9c01-f7c68229a99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:29.673215 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:29.678838 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 497/981] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:29 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:29.692647 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d75cb7-6341-4aff-b0c8-41e28a940d92 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:29.702311 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceadb0b-d5e6-417a-9c01-f7c68229a99c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:29.747389 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6eaf24-f3d7-4285-a3c6-5394324c0283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:29.748553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6eaf24-f3d7-4285-a3c6-5394324c0283 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:29.749638 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6eaf24-f3d7-4285-a3c6-5394324c0283 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:29.750113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6eaf24-f3d7-4285-a3c6-5394324c0283 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:29.764463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6eaf24-f3d7-4285-a3c6-5394324c0283 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:29.766899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6eaf24-f3d7-4285-a3c6-5394324c0283 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:29.793476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-2d6eaf24-f3d7-4285-a3c6-5394324c0283 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:29.793476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:29.793476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:29.793476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:29.793476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:29.793476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:29.793476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:29.793476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:29.793476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:29.793476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:29.793476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:29.793476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:29.793476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:29.793476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:29.793476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:29.794483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:29.794483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:29.794483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:29.794483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:29.794483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:29.794483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:29.794483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:29.794483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:29.794483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:29.794483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:29.794483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:29.794483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:29.794483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:29.794483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:29.794483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:29.794483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:29.794483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:29.795406 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:29.795406 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:29.795406 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:29.795406 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:29.796405 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 485/982] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:29.803489 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6eaf24-f3d7-4285-a3c6-5394324c0283 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:29.825011 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:29.887355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceadb0b-d5e6-417a-9c01-f7c68229a99c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:29.950384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974d5d0-b78e-49b0-b7aa-67138e89fd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:29.951612 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974d5d0-b78e-49b0-b7aa-67138e89fd67 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:29.952104 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974d5d0-b78e-49b0-b7aa-67138e89fd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:29.952566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974d5d0-b78e-49b0-b7aa-67138e89fd67 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:29.953658 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974d5d0-b78e-49b0-b7aa-67138e89fd67 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:29.956238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974d5d0-b78e-49b0-b7aa-67138e89fd67 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:29.974779 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6eaf24-f3d7-4285-a3c6-5394324c0283 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:29.981545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-3974d5d0-b78e-49b0-b7aa-67138e89fd67 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 08:26:29.981545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:29.981545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:26:29.981545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:26:29.981545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:26:29.981545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:26:29.981545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:26:29.981545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:29.981545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 08:26:29.981545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:29.981545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:29.981545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:29.981545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:29.981545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:29.981545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:29.981545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:29.981545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:29.982923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:29.982923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:29.982923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:29.982923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:29.982923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:29.982923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:29.982923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:26:29.982923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:26:29.982923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:26:29.982923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:26:29.982923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:29.982923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:29.982923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:29.982923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:29.982923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:29.982923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:29.984362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:29.984362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:29.984362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:29.984362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:29.984362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:29.984362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:29.984362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:29.984362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:29.984362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:29.984362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:29.984362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:26:29.984362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:29.984362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 08:26:29.984362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:29.984362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 498/983] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:26:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:29.991488 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974d5d0-b78e-49b0-b7aa-67138e89fd67 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:30.036638 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065bd809-8aca-4b25-883f-b6e880a728b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:30.038086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bd809-8aca-4b25-883f-b6e880a728b7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:30.038762 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bd809-8aca-4b25-883f-b6e880a728b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:30.039452 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bd809-8aca-4b25-883f-b6e880a728b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:30.041003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065bd809-8aca-4b25-883f-b6e880a728b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:30.041783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065bd809-8aca-4b25-883f-b6e880a728b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:30.044574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065bd809-8aca-4b25-883f-b6e880a728b7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:30.068927 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 486/984] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:30.075106 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065bd809-8aca-4b25-883f-b6e880a728b7 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:30.082019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:30.156597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974d5d0-b78e-49b0-b7aa-67138e89fd67 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:30.196561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb353243-9d3e-4a04-b5f5-f13f95eb1a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:30.197224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb353243-9d3e-4a04-b5f5-f13f95eb1a61 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:30.197506 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb353243-9d3e-4a04-b5f5-f13f95eb1a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:30.197783 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb353243-9d3e-4a04-b5f5-f13f95eb1a61 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:30.198491 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb353243-9d3e-4a04-b5f5-f13f95eb1a61 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:30.198797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb353243-9d3e-4a04-b5f5-f13f95eb1a61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:30.200228 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb353243-9d3e-4a04-b5f5-f13f95eb1a61 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:30.201486 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:30.207738 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 499/985] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 18 08:26:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:30.209070 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abff08-96c4-4a0a-835a-2c8a2a11ca55 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:30.210635 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abff08-96c4-4a0a-835a-2c8a2a11ca55 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:30.210635 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abff08-96c4-4a0a-835a-2c8a2a11ca55 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:30.211347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 500/986] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:30.216213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abff08-96c4-4a0a-835a-2c8a2a11ca55 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:30.224058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065bd809-8aca-4b25-883f-b6e880a728b7 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:30.266437 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1182baf-67a6-42e2-a69b-f7d92ed88d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:30.266970 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1182baf-67a6-42e2-a69b-f7d92ed88d97 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:30.267295 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1182baf-67a6-42e2-a69b-f7d92ed88d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:30.267735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1182baf-67a6-42e2-a69b-f7d92ed88d97 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:30.276849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1182baf-67a6-42e2-a69b-f7d92ed88d97 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:30.278195 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1182baf-67a6-42e2-a69b-f7d92ed88d97 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:30.294746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-b1182baf-67a6-42e2-a69b-f7d92ed88d97 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 08:26:30.294746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:30.294746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:30.294746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:30.294746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:30.294746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:30.294746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:30.294746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:30.294746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:30.294746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:30.294746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:30.294746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:30.294746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:26:30.294746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:26:30.294746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:26:30.294746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:26:30.296751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:30.296751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:30.296751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:26:30.296751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:30.296751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:26:30.296751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:30.296751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:30.296751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:30.296751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:30.296751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:30.296751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:30.296751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:30.296751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:30.296751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:30.296751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:30.296751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:30.296751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:30.296751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:30.298541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:30.298541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:30.298541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:26:30.298541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:26:30.298541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:26:30.298541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:26:30.298541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:26:30.298541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:26:30.298541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 08:26:30.298541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:30.298541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 487/987] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 18 08:26:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:30.302494 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984c785-632c-4dd5-be62-b8f206b153d5 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:30.302840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984c785-632c-4dd5-be62-b8f206b153d5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:30.303121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984c785-632c-4dd5-be62-b8f206b153d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:30.323204 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-9984c785-632c-4dd5-be62-b8f206b153d5 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:30.324577 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9984c785-632c-4dd5-be62-b8f206b153d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:30.345529 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abff08-96c4-4a0a-835a-2c8a2a11ca55 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:30.394090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ffd45-fdf6-4e91-ae4e-af440e85bd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:30.395203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ffd45-fdf6-4e91-ae4e-af440e85bd90 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:30.395661 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ffd45-fdf6-4e91-ae4e-af440e85bd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:30.396155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ffd45-fdf6-4e91-ae4e-af440e85bd90 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:30.397328 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ffd45-fdf6-4e91-ae4e-af440e85bd90 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:30.397837 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ffd45-fdf6-4e91-ae4e-af440e85bd90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:30.400389 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ffd45-fdf6-4e91-ae4e-af440e85bd90 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:30.409168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 501/988] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 18 08:26:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:30.410664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ffd45-fdf6-4e91-ae4e-af440e85bd90 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:30.413208 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:30.424019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 488/989] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:30 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:30.431381 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984c785-632c-4dd5-be62-b8f206b153d5 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:30.522069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:30.570922 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ffd45-fdf6-4e91-ae4e-af440e85bd90 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:30.582008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984c785-632c-4dd5-be62-b8f206b153d5 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:30.627253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452c14e-b140-49d4-8a77-8edf4052b9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:30.627969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452c14e-b140-49d4-8a77-8edf4052b9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:30.629043 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452c14e-b140-49d4-8a77-8edf4052b9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:30.629416 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452c14e-b140-49d4-8a77-8edf4052b9d0 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:30.630563 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d452c14e-b140-49d4-8a77-8edf4052b9d0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:30.630878 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d452c14e-b140-49d4-8a77-8edf4052b9d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:30.632377 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d452c14e-b140-49d4-8a77-8edf4052b9d0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:30.636355 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93efca8b-9dd7-43c0-8650-1588114c309d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:30.637091 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efca8b-9dd7-43c0-8650-1588114c309d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:30.637375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efca8b-9dd7-43c0-8650-1588114c309d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:30.637650 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efca8b-9dd7-43c0-8650-1588114c309d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:30.638455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93efca8b-9dd7-43c0-8650-1588114c309d None admin] RBAC: Adding query filter params (interface=public, service_id=20f4d59a690940cb94f37a2c6cd00ae9, region_id=RegionOne) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:30.638765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93efca8b-9dd7-43c0-8650-1588114c309d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:30.640178 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93efca8b-9dd7-43c0-8650-1588114c309d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:30.650020 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 489/990] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 18 08:26:30 2022] GET /identity/v3/endpoints?service_id=20f4d59a690940cb94f37a2c6cd00ae9&interface=public®ion_id=RegionOne => generated 194 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:30.652031 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93efca8b-9dd7-43c0-8650-1588114c309d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:30.653690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 502/991] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Jul 18 08:26:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:30.663040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452c14e-b140-49d4-8a77-8edf4052b9d0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:30.725763 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:30.774188 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93efca8b-9dd7-43c0-8650-1588114c309d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:30.787836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452c14e-b140-49d4-8a77-8edf4052b9d0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:30.811138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:30.817572 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b09188-f7ba-4990-8d03-00b23ec73a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:30.818450 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b09188-f7ba-4990-8d03-00b23ec73a1f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:30.818735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b09188-f7ba-4990-8d03-00b23ec73a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:30.825768 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b09188-f7ba-4990-8d03-00b23ec73a1f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:30.825768 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b09188-f7ba-4990-8d03-00b23ec73a1f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:30.825768 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b09188-f7ba-4990-8d03-00b23ec73a1f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:30.827278 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2617c-92db-49a5-b49a-5fcb327684c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:30.828064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2617c-92db-49a5-b49a-5fcb327684c5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:30.828352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2617c-92db-49a5-b49a-5fcb327684c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:30.828632 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2617c-92db-49a5-b49a-5fcb327684c5 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:30.829336 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2617c-92db-49a5-b49a-5fcb327684c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:30.830761 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2617c-92db-49a5-b49a-5fcb327684c5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:30.835599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-f2b09188-f7ba-4990-8d03-00b23ec73a1f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 08:26:30.835599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:30.835599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:26:30.835599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:26:30.835599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:26:30.835599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:26:30.835599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:26:30.835599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:30.835599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 08:26:30.835599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:30.835599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:30.835599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:30.835599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:30.835599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:30.835599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:30.835599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:30.835599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:30.840175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:30.840175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:30.840175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:30.840175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:30.840175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:30.840175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:30.840175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:26:30.840175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:26:30.840175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:26:30.840175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:26:30.840175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:30.840175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:30.840175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:30.840175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:30.840175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:30.840175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:30.841567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:30.841567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:30.841567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:30.841567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:30.841567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:30.841567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:30.841567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:30.841567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:30.841567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:30.841567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:30.841567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:26:30.841567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:30.841567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 08:26:30.841567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:30.841567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 490/992] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Jul 18 08:26:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:30.842504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 503/993] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 18 08:26:30 2022] GET /identity/v3/services => generated 1727 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:26:30.842581 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b09188-f7ba-4990-8d03-00b23ec73a1f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:30.845172 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2617c-92db-49a5-b49a-5fcb327684c5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:31.016375 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2617c-92db-49a5-b49a-5fcb327684c5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:31.021040 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b09188-f7ba-4990-8d03-00b23ec73a1f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:31.049170 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:31.073864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307344e7-350a-4c89-8ae8-d004d8fc2ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:31.074841 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8161f4-9fba-4932-bf73-82afb6bead95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:31.075066 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307344e7-350a-4c89-8ae8-d004d8fc2ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:31.075363 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307344e7-350a-4c89-8ae8-d004d8fc2ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:31.075521 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8161f4-9fba-4932-bf73-82afb6bead95 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:31.075972 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307344e7-350a-4c89-8ae8-d004d8fc2ff3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:31.076145 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8161f4-9fba-4932-bf73-82afb6bead95 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:31.076434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8161f4-9fba-4932-bf73-82afb6bead95 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:31.077214 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8161f4-9fba-4932-bf73-82afb6bead95 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:31.077532 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8161f4-9fba-4932-bf73-82afb6bead95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:31.079138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8161f4-9fba-4932-bf73-82afb6bead95 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:31.088784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 504/994] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 18 08:26:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:31.098568 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8161f4-9fba-4932-bf73-82afb6bead95 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:31.191411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:31.203585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307344e7-350a-4c89-8ae8-d004d8fc2ff3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:31.206044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307344e7-350a-4c89-8ae8-d004d8fc2ff3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:31.261357 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8161f4-9fba-4932-bf73-82afb6bead95 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:31.304202 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d3b71-c7eb-4dbd-b599-48c399179132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:31.304877 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d3b71-c7eb-4dbd-b599-48c399179132 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:31.305161 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d3b71-c7eb-4dbd-b599-48c399179132 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:31.305437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d3b71-c7eb-4dbd-b599-48c399179132 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:31.306153 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d3b71-c7eb-4dbd-b599-48c399179132 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:31.306461 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d3b71-c7eb-4dbd-b599-48c399179132 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:31.307864 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d3b71-c7eb-4dbd-b599-48c399179132 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:31.313352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 505/995] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Jul 18 08:26:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:31.319428 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d3b71-c7eb-4dbd-b599-48c399179132 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:31.380540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 491/996] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:26:30 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 08:26:31.381494 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307344e7-350a-4c89-8ae8-d004d8fc2ff3 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:31.413661 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:31.452675 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d3b71-c7eb-4dbd-b599-48c399179132 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:31.473459 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:31.505246 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdf6c8-e77c-43a4-bcf9-3cedd1a1df5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:31.506130 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdf6c8-e77c-43a4-bcf9-3cedd1a1df5b None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:31.506601 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdf6c8-e77c-43a4-bcf9-3cedd1a1df5b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:31.506976 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdf6c8-e77c-43a4-bcf9-3cedd1a1df5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:31.507965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfdf6c8-e77c-43a4-bcf9-3cedd1a1df5b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:31.509751 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfdf6c8-e77c-43a4-bcf9-3cedd1a1df5b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:31.519552 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307344e7-350a-4c89-8ae8-d004d8fc2ff3 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:31.562115 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab308e-8f90-48c9-a0ec-32bd7c79c9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:31.562925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab308e-8f90-48c9-a0ec-32bd7c79c9cf None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:31.563206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab308e-8f90-48c9-a0ec-32bd7c79c9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:31.563487 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab308e-8f90-48c9-a0ec-32bd7c79c9cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:31.574930 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab308e-8f90-48c9-a0ec-32bd7c79c9cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:31.576440 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab308e-8f90-48c9-a0ec-32bd7c79c9cf None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:31.593042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 506/997] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 18 08:26:31 2022] POST /identity/v3/endpoints => generated 325 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:26:31.598685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-bdab308e-8f90-48c9-a0ec-32bd7c79c9cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:31.598685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:31.598685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:31.598685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:31.598685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:31.598685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:31.598685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:31.598685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:31.598685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:31.598685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:31.598685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:31.598685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:31.598685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:31.598685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:31.598685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:31.600830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:31.600830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:31.600830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:31.600830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:31.600830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:31.600830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:31.600830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:31.600830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:31.600830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:31.600830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:31.600830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:31.600830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:31.600830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:31.600830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:31.600830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:31.600830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:31.600830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:31.602742 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:31.602742 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:31.602742 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:31.602742 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:31.604793 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 492/998] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:31.606874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdf6c8-e77c-43a4-bcf9-3cedd1a1df5b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:31.802486 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdf6c8-e77c-43a4-bcf9-3cedd1a1df5b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:31.844971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90659320-ee05-4fe9-bd9a-f38116f7c7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:31.844971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90659320-ee05-4fe9-bd9a-f38116f7c7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:31.844971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90659320-ee05-4fe9-bd9a-f38116f7c7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:31.844971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90659320-ee05-4fe9-bd9a-f38116f7c7f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:31.845341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90659320-ee05-4fe9-bd9a-f38116f7c7f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:31.845341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90659320-ee05-4fe9-bd9a-f38116f7c7f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:31.845927 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90659320-ee05-4fe9-bd9a-f38116f7c7f6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:31.863726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 507/999] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:31.865963 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab308e-8f90-48c9-a0ec-32bd7c79c9cf None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:31.946171 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:31.972643 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab308e-8f90-48c9-a0ec-32bd7c79c9cf None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:32.008394 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c6ae0-bf64-4c29-adac-e38fa6e1516f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:32.009205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c6ae0-bf64-4c29-adac-e38fa6e1516f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:32.009482 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c6ae0-bf64-4c29-adac-e38fa6e1516f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:32.009758 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c6ae0-bf64-4c29-adac-e38fa6e1516f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:32.016961 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c6ae0-bf64-4c29-adac-e38fa6e1516f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:32.018430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c6ae0-bf64-4c29-adac-e38fa6e1516f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:32.033857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-ef3c6ae0-bf64-4c29-adac-e38fa6e1516f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:32.033857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:32.033857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:32.033857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:32.033857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:32.033857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:32.033857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:32.033857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:32.033857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:32.033857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:32.033857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:32.033857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:32.033857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:26:32.033857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:26:32.033857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:26:32.035403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:26:32.035403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:32.035403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:32.035403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:32.035403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:32.035403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:32.035403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:32.035403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:32.035403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:32.035403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:32.035403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:32.035403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:32.035403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:32.035403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:32.035403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:32.035403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:32.035403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:32.041800 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:26:32.041800 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:26:32.041800 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:26:32.041800 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:26:32.041800 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:26:32.041800 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:26:32.041800 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:32.041800 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:32.041800 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 493/1000] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:26:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:32.042310 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90659320-ee05-4fe9-bd9a-f38116f7c7f6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:32.078698 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:32.165073 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90659320-ee05-4fe9-bd9a-f38116f7c7f6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:32.205579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9b5ec-f284-4448-8bca-749410d9fb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:32.206986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9b5ec-f284-4448-8bca-749410d9fb61 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:32.207156 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9b5ec-f284-4448-8bca-749410d9fb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:32.207532 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9b5ec-f284-4448-8bca-749410d9fb61 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:32.208404 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9b5ec-f284-4448-8bca-749410d9fb61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:32.208739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9b5ec-f284-4448-8bca-749410d9fb61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:32.210252 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9b5ec-f284-4448-8bca-749410d9fb61 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:32.226023 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 508/1001] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:26:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:32.232126 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c6ae0-bf64-4c29-adac-e38fa6e1516f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:32.352657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c6ae0-bf64-4c29-adac-e38fa6e1516f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:32.392343 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67346e2f-57f1-4b49-b972-a2b654430d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:32.393007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67346e2f-57f1-4b49-b972-a2b654430d8f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:32.393290 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67346e2f-57f1-4b49-b972-a2b654430d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:32.393568 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67346e2f-57f1-4b49-b972-a2b654430d8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:32.394947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67346e2f-57f1-4b49-b972-a2b654430d8f None admin] RBAC: Adding query filter params (role.id=9fd88609ddd942d5a9b4bc684e5e49b2, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=2b03f8b8c51e472bbd4cf0afc9c6b632) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:32.395262 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67346e2f-57f1-4b49-b972-a2b654430d8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:32.396867 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67346e2f-57f1-4b49-b972-a2b654430d8f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:32.399071 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:32.405418 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 494/1002] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:26:32 2022] GET /identity/v3/role_assignments?user.id=2b03f8b8c51e472bbd4cf0afc9c6b632&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=9fd88609ddd942d5a9b4bc684e5e49b2 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:33.077342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f54b2-ef3e-440a-9fdf-b9421a83233c None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:33.078128 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f54b2-ef3e-440a-9fdf-b9421a83233c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:33.078300 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f54b2-ef3e-440a-9fdf-b9421a83233c None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:33.081058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 509/1003] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:33.086419 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4827934-e0de-4e20-b586-b82effffc70c None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:33.087190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4827934-e0de-4e20-b586-b82effffc70c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:33.087487 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4827934-e0de-4e20-b586-b82effffc70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:33.106386 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d4827934-e0de-4e20-b586-b82effffc70c None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:33.107736 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d4827934-e0de-4e20-b586-b82effffc70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:33.225048 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 495/1004] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:33 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:33.257193 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f54b2-ef3e-440a-9fdf-b9421a83233c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:33.427469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:33.453882 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f54b2-ef3e-440a-9fdf-b9421a83233c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:33.520731 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ead2c-e0ed-4de7-83f9-a06d912cbdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:33.522116 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ead2c-e0ed-4de7-83f9-a06d912cbdb0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:33.522614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ead2c-e0ed-4de7-83f9-a06d912cbdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:33.523107 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ead2c-e0ed-4de7-83f9-a06d912cbdb0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:33.524411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ead2c-e0ed-4de7-83f9-a06d912cbdb0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:33.527000 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ead2c-e0ed-4de7-83f9-a06d912cbdb0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:33.551326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-f09ead2c-e0ed-4de7-83f9-a06d912cbdb0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 08:26:33.551326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:33.551326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:26:33.551326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:26:33.551326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:26:33.551326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:26:33.551326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:26:33.551326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:33.551326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 08:26:33.551326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:33.551326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:33.551326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:33.551326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:33.551326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:33.551326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:33.551326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:33.551326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:33.552819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:33.552819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:33.552819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:33.552819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:33.552819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:33.552819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:33.552819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:26:33.552819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:26:33.552819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:26:33.552819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:26:33.552819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:33.552819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:33.552819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:33.552819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:33.552819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:33.552819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:33.561620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:33.561620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:33.561620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:33.561620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:33.561620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:33.561620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:33.561620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:33.561620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:33.561620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:33.561620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:33.561620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:26:33.561620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:33.561620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 08:26:33.561620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:33.561620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 510/1005] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:26:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:33.567921 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4827934-e0de-4e20-b586-b82effffc70c None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:33.695271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4827934-e0de-4e20-b586-b82effffc70c None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:33.731569 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:33.744427 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43eec44-487c-484d-b4ad-52703b778e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:33.745233 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43eec44-487c-484d-b4ad-52703b778e45 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:33.745679 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43eec44-487c-484d-b4ad-52703b778e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:33.745965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43eec44-487c-484d-b4ad-52703b778e45 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:33.746701 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43eec44-487c-484d-b4ad-52703b778e45 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:33.747173 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43eec44-487c-484d-b4ad-52703b778e45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:33.748773 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43eec44-487c-484d-b4ad-52703b778e45 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:33.758073 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 496/1006] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 18 08:26:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:33.765618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ead2c-e0ed-4de7-83f9-a06d912cbdb0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:33.958455 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ead2c-e0ed-4de7-83f9-a06d912cbdb0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:33.965167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2e65b-2b0e-4ede-81bd-8e3f7002b7d5 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:33.965564 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2e65b-2b0e-4ede-81bd-8e3f7002b7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:33.965727 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2e65b-2b0e-4ede-81bd-8e3f7002b7d5 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:33.967331 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 497/1007] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:26:33.973301 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ad68c-0494-4d2c-8555-4f111a9ba939 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:33.973597 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ad68c-0494-4d2c-8555-4f111a9ba939 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:33.973872 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ad68c-0494-4d2c-8555-4f111a9ba939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:33.992148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-7f2ad68c-0494-4d2c-8555-4f111a9ba939 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:33.993344 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7f2ad68c-0494-4d2c-8555-4f111a9ba939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:34.022893 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea0455-20da-42a7-b0d0-432d0a7e0626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:34.024111 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea0455-20da-42a7-b0d0-432d0a7e0626 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:34.024111 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea0455-20da-42a7-b0d0-432d0a7e0626 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:34.024431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea0455-20da-42a7-b0d0-432d0a7e0626 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:34.025086 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea0455-20da-42a7-b0d0-432d0a7e0626 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:34.025337 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea0455-20da-42a7-b0d0-432d0a7e0626 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:34.026735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea0455-20da-42a7-b0d0-432d0a7e0626 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:34.033207 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 511/1008] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Jul 18 08:26:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:34.040000 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea0455-20da-42a7-b0d0-432d0a7e0626 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:34.065612 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:34.092471 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 498/1009] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:33 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:34.108385 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ad68c-0494-4d2c-8555-4f111a9ba939 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:34.185914 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea0455-20da-42a7-b0d0-432d0a7e0626 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:34.246728 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70945441-a2de-401a-bf1e-d096b8893d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:34.247513 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70945441-a2de-401a-bf1e-d096b8893d82 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:34.248039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70945441-a2de-401a-bf1e-d096b8893d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:34.248347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70945441-a2de-401a-bf1e-d096b8893d82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:34.249611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70945441-a2de-401a-bf1e-d096b8893d82 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:34.251628 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70945441-a2de-401a-bf1e-d096b8893d82 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:34.264309 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ad68c-0494-4d2c-8555-4f111a9ba939 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:34.291157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:34.304660 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1abcc0-7384-426c-ad64-1c5411cd5eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:34.305469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1abcc0-7384-426c-ad64-1c5411cd5eba None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:34.305753 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1abcc0-7384-426c-ad64-1c5411cd5eba None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:34.306061 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1abcc0-7384-426c-ad64-1c5411cd5eba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:34.307486 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1abcc0-7384-426c-ad64-1c5411cd5eba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:34.309033 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1abcc0-7384-426c-ad64-1c5411cd5eba None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:34.311007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 499/1010] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:34.317276 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1abcc0-7384-426c-ad64-1c5411cd5eba None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:34.342243 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 512/1011] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 18 08:26:34 2022] POST /identity/v3/endpoints => generated 334 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:26:34.513295 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1abcc0-7384-426c-ad64-1c5411cd5eba None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:34.578296 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef1bd4-1241-41a2-a7d2-af8a97b8bfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:34.579195 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef1bd4-1241-41a2-a7d2-af8a97b8bfe5 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:34.579475 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef1bd4-1241-41a2-a7d2-af8a97b8bfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:34.579839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef1bd4-1241-41a2-a7d2-af8a97b8bfe5 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:34.580576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ef1bd4-1241-41a2-a7d2-af8a97b8bfe5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:34.582061 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ef1bd4-1241-41a2-a7d2-af8a97b8bfe5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:34.622352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5269aa-c7a5-4b5f-bfd4-0ce2204ff0ea None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:34.622652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5269aa-c7a5-4b5f-bfd4-0ce2204ff0ea None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:34.622943 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5269aa-c7a5-4b5f-bfd4-0ce2204ff0ea None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:34.624472 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 513/1012] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:34.631290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8379f-cca2-4c32-8747-8aef5d8558b7 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:34.631558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8379f-cca2-4c32-8747-8aef5d8558b7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:34.631832 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8379f-cca2-4c32-8747-8aef5d8558b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:34.651725 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-cbd8379f-cca2-4c32-8747-8aef5d8558b7 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:34.652948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cbd8379f-cca2-4c32-8747-8aef5d8558b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:34.702140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 500/1013] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 18 08:26:34 2022] POST /identity/v3/projects => generated 311 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:26:34.751706 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 514/1014] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:34 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:34.786777 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef1bd4-1241-41a2-a7d2-af8a97b8bfe5 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:34.856475 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:34.936930 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef1bd4-1241-41a2-a7d2-af8a97b8bfe5 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:34.978926 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656ba7e-96c4-4db3-bc53-2e5bbdbb3581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:34.979778 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656ba7e-96c4-4db3-bc53-2e5bbdbb3581 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:34.980069 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656ba7e-96c4-4db3-bc53-2e5bbdbb3581 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:34.980346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656ba7e-96c4-4db3-bc53-2e5bbdbb3581 None admin] PATH_INFO: `/v3/services/image` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:34.981016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656ba7e-96c4-4db3-bc53-2e5bbdbb3581 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:34.982402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656ba7e-96c4-4db3-bc53-2e5bbdbb3581 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:34.995074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-c656ba7e-96c4-4db3-bc53-2e5bbdbb3581 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 08:26:34.995074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:34.995074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:26:34.995074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:26:34.995074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:26:34.995074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:26:34.995074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:26:34.995074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:34.995074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 08:26:34.995074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:34.995074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:34.995074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:34.995074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:34.995074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:34.995074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:34.995074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:34.995074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:34.997019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:34.997019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:34.997019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:34.997019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:34.997019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:34.997019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:34.997019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:26:34.997019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:26:34.997019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:26:34.997019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:26:34.997019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:34.997019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:34.997019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:34.997019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:34.997019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:34.997019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:34.998547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:34.998547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:34.998547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:34.998547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:34.998547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:34.998547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:34.998547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:34.998547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:34.998547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:34.998547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:34.998547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:26:34.998547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:34.998547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 08:26:34.998547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:34.998547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 501/1015] 10.222.0.57 () {56 vars in 1105 bytes} [Mon Jul 18 08:26:34 2022] GET /identity/v3/services/image => generated 86 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:35.002506 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8379f-cca2-4c32-8747-8aef5d8558b7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:35.005837 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:35.126897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8379f-cca2-4c32-8747-8aef5d8558b7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:35.166464 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb93a9c-0750-45a7-b7ac-2eaa8e02ee05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:35.167168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb93a9c-0750-45a7-b7ac-2eaa8e02ee05 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:35.167435 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb93a9c-0750-45a7-b7ac-2eaa8e02ee05 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:35.167737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb93a9c-0750-45a7-b7ac-2eaa8e02ee05 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:35.168478 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb93a9c-0750-45a7-b7ac-2eaa8e02ee05 None admin] RBAC: Adding query filter params (name=image) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:35.168768 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb93a9c-0750-45a7-b7ac-2eaa8e02ee05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:35.170154 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb93a9c-0750-45a7-b7ac-2eaa8e02ee05 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:35.175983 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 515/1016] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:35.182153 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656ba7e-96c4-4db3-bc53-2e5bbdbb3581 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:35.295521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656ba7e-96c4-4db3-bc53-2e5bbdbb3581 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:35.297325 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:35.334031 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d51ba-d740-4e6d-905c-e10d918368f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:35.334693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d51ba-d740-4e6d-905c-e10d918368f7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:35.334973 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d51ba-d740-4e6d-905c-e10d918368f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:35.335248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d51ba-d740-4e6d-905c-e10d918368f7 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:35.335988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d51ba-d740-4e6d-905c-e10d918368f7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:35.336295 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d51ba-d740-4e6d-905c-e10d918368f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:35.337650 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d51ba-d740-4e6d-905c-e10d918368f7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:35.347940 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 502/1017] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:35 2022] GET /identity/v3/services?type=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:36.850467 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652baefd-2315-419b-84a7-ff97c9edd39d None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:36.851112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652baefd-2315-419b-84a7-ff97c9edd39d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:36.851112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652baefd-2315-419b-84a7-ff97c9edd39d None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:36.852620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 516/1018] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:26:36.857839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bca14-8a8f-4ffc-9bab-ec497efd20e7 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:36.858281 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bca14-8a8f-4ffc-9bab-ec497efd20e7 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:36.858461 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bca14-8a8f-4ffc-9bab-ec497efd20e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:36.877098 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-076bca14-8a8f-4ffc-9bab-ec497efd20e7 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:36.879837 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-076bca14-8a8f-4ffc-9bab-ec497efd20e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:36.963719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 503/1019] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:36 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:36.986451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652baefd-2315-419b-84a7-ff97c9edd39d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:37.068504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:37.123097 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652baefd-2315-419b-84a7-ff97c9edd39d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:37.131248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cd52b-e117-48f4-b337-5b58231d6c2b None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:37.131874 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cd52b-e117-48f4-b337-5b58231d6c2b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:37.132273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cd52b-e117-48f4-b337-5b58231d6c2b None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:37.134395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 504/1020] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:26:37.140748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f0185-4c53-4d8a-85be-8b3fe459774d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:37.141189 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f0185-4c53-4d8a-85be-8b3fe459774d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:37.141656 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f0185-4c53-4d8a-85be-8b3fe459774d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:37.164845 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6f5f0185-4c53-4d8a-85be-8b3fe459774d None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:37.166998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6f5f0185-4c53-4d8a-85be-8b3fe459774d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:37.173679 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781152c-eac9-4911-88e6-a59f273d0cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:37.175496 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781152c-eac9-4911-88e6-a59f273d0cae None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:37.175958 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781152c-eac9-4911-88e6-a59f273d0cae None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:37.176378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781152c-eac9-4911-88e6-a59f273d0cae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:37.177759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781152c-eac9-4911-88e6-a59f273d0cae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:37.179229 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781152c-eac9-4911-88e6-a59f273d0cae None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:37.181145 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 517/1021] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:37.186864 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781152c-eac9-4911-88e6-a59f273d0cae None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:37.261352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 505/1022] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:37 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:37.286792 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f0185-4c53-4d8a-85be-8b3fe459774d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:37.325989 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:37.348422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781152c-eac9-4911-88e6-a59f273d0cae None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:37.408649 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949f692-b0c7-4c24-821f-2c85e502c8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:37.410192 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949f692-b0c7-4c24-821f-2c85e502c8e5 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:37.410620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949f692-b0c7-4c24-821f-2c85e502c8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:37.411043 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949f692-b0c7-4c24-821f-2c85e502c8e5 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:37.411988 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949f692-b0c7-4c24-821f-2c85e502c8e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:37.413620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949f692-b0c7-4c24-821f-2c85e502c8e5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:37.420886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:37.431079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f0185-4c53-4d8a-85be-8b3fe459774d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:37.485069 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f5989-7360-4d97-a7bf-4f2058f0c363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:37.485889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f5989-7360-4d97-a7bf-4f2058f0c363 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:37.486171 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f5989-7360-4d97-a7bf-4f2058f0c363 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:37.486444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f5989-7360-4d97-a7bf-4f2058f0c363 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:37.498482 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f5989-7360-4d97-a7bf-4f2058f0c363 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:26:37.506925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-2b6f5989-7360-4d97-a7bf-4f2058f0c363 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 08:26:37.506925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:37.506925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:37.506925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:37.506925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:37.506925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:37.506925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:37.506925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:37.506925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:37.506925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:37.506925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:37.506925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:37.506925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:26:37.506925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:26:37.506925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:26:37.506925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:26:37.508659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:26:37.508659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:26:37.508659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:26:37.508659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:26:37.508659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:37.508659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:37.508659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:37.508659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:37.508659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:37.508659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:37.508659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:37.508659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:37.508659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:37.508659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:37.508659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:37.508659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:37.509641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:37.509641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:37.509641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:37.509641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:37.509641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:26:37.509641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:26:37.509641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:26:37.509641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:26:37.509641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:26:37.509641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:26:37.509641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 08:26:37.509641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:37.509641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 506/1023] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Jul 18 08:26:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:37.509641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cc03f-7582-4107-aff5-8baf6f1030e5 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:37.510472 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cc03f-7582-4107-aff5-8baf6f1030e5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:37.510472 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cc03f-7582-4107-aff5-8baf6f1030e5 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:37.511515 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 507/1024] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:37.514792 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cc03f-7582-4107-aff5-8baf6f1030e5 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:37.524681 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 518/1025] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 18 08:26:37 2022] POST /identity/v3/users => generated 285 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:26:37.525844 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d606063-c534-464f-9455-8472f42abed0 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:37.526098 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d606063-c534-464f-9455-8472f42abed0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:37.526328 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d606063-c534-464f-9455-8472f42abed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:37.546388 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-2d606063-c534-464f-9455-8472f42abed0 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:37.547659 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2d606063-c534-464f-9455-8472f42abed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:37.637434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 519/1026] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:37 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:37.662467 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d606063-c534-464f-9455-8472f42abed0 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:37.683240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:37.704125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cc03f-7582-4107-aff5-8baf6f1030e5 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:37.765095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456a693-5820-47ac-b66f-01b595319cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:37.765805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456a693-5820-47ac-b66f-01b595319cff None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:37.766098 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456a693-5820-47ac-b66f-01b595319cff None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:37.766379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456a693-5820-47ac-b66f-01b595319cff None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:37.767186 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456a693-5820-47ac-b66f-01b595319cff None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:37.767634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456a693-5820-47ac-b66f-01b595319cff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:37.769645 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456a693-5820-47ac-b66f-01b595319cff None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:37.780051 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 508/1027] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 18 08:26:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:37.789127 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456a693-5820-47ac-b66f-01b595319cff None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:37.826528 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d606063-c534-464f-9455-8472f42abed0 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:37.868330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:37.889466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593f344-a214-4e17-a9c0-ee8f59e15576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:37.890153 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593f344-a214-4e17-a9c0-ee8f59e15576 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:37.890468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593f344-a214-4e17-a9c0-ee8f59e15576 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:37.890760 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593f344-a214-4e17-a9c0-ee8f59e15576 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:37.891648 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593f344-a214-4e17-a9c0-ee8f59e15576 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:37.893238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593f344-a214-4e17-a9c0-ee8f59e15576 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:37.981514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 520/1028] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 18 08:26:37 2022] POST /identity/v3/services => generated 242 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:26:38.042478 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456a693-5820-47ac-b66f-01b595319cff None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:38.104939 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad900f-ae59-48c1-83b0-e227ba1f3ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:38.106311 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad900f-ae59-48c1-83b0-e227ba1f3ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:38.106827 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad900f-ae59-48c1-83b0-e227ba1f3ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:38.107507 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad900f-ae59-48c1-83b0-e227ba1f3ce3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:38.139769 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad900f-ae59-48c1-83b0-e227ba1f3ce3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:38.144271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad900f-ae59-48c1-83b0-e227ba1f3ce3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:38.150627 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:38.178875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-74ad900f-ae59-48c1-83b0-e227ba1f3ce3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:38.178875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:38.178875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:38.178875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:38.178875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:38.178875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:38.178875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:38.178875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:38.178875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:38.178875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:38.178875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:38.178875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:38.178875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:38.178875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:38.178875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:38.181324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:38.181324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:38.181324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:38.181324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:38.181324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:38.181324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:38.181324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:38.181324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:38.181324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:38.181324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:38.181324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:38.181324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:38.181324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:38.181324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:38.181324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:38.181324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:38.181324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:38.182935 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:38.182935 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:38.182935 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:38.182935 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:38.182935 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 509/1029] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:38.189785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593f344-a214-4e17-a9c0-ee8f59e15576 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:38.346754 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593f344-a214-4e17-a9c0-ee8f59e15576 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:38.412523 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c13326-70f7-40fe-835e-a81f230bbce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:38.413684 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c13326-70f7-40fe-835e-a81f230bbce4 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:38.413974 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c13326-70f7-40fe-835e-a81f230bbce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:38.414405 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c13326-70f7-40fe-835e-a81f230bbce4 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:38.415837 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c13326-70f7-40fe-835e-a81f230bbce4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:38.416371 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c13326-70f7-40fe-835e-a81f230bbce4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:38.419055 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c13326-70f7-40fe-835e-a81f230bbce4 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:38.448992 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 521/1030] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:38.450865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad900f-ae59-48c1-83b0-e227ba1f3ce3 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:38.616803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad900f-ae59-48c1-83b0-e227ba1f3ce3 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:38.674462 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18feb7a-3014-407e-be7e-e925abb6e33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:38.676455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18feb7a-3014-407e-be7e-e925abb6e33e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:38.676958 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18feb7a-3014-407e-be7e-e925abb6e33e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:38.677454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18feb7a-3014-407e-be7e-e925abb6e33e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:38.692916 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18feb7a-3014-407e-be7e-e925abb6e33e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:38.695731 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18feb7a-3014-407e-be7e-e925abb6e33e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:38.725012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-f18feb7a-3014-407e-be7e-e925abb6e33e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 08:26:38.725012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:38.725012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:38.725012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:38.725012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:38.725012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:38.725012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:38.725012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:38.725012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:38.725012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:38.725012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:38.725012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:38.725012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:26:38.725012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:26:38.725012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:26:38.725012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:26:38.726229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:38.726229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:38.726229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:26:38.726229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:38.726229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:26:38.726229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:38.726229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:38.726229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:38.726229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:38.726229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:38.726229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:38.726229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:38.726229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:38.726229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:38.726229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:38.726229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:38.726229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:38.726229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:38.735161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:38.735161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:38.735161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:26:38.735161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:26:38.735161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:26:38.735161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:26:38.735161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:26:38.735161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:26:38.735161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 08:26:38.735161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:38.735161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 510/1031] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 18 08:26:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:38.736163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c13326-70f7-40fe-835e-a81f230bbce4 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:38.801843 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:38.873618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c13326-70f7-40fe-835e-a81f230bbce4 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:38.912141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce42859-8f8d-4ca8-aa18-4b88f168a355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:38.912835 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce42859-8f8d-4ca8-aa18-4b88f168a355 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:38.913116 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce42859-8f8d-4ca8-aa18-4b88f168a355 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:38.913400 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce42859-8f8d-4ca8-aa18-4b88f168a355 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:38.914261 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce42859-8f8d-4ca8-aa18-4b88f168a355 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:38.914577 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce42859-8f8d-4ca8-aa18-4b88f168a355 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:38.916053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce42859-8f8d-4ca8-aa18-4b88f168a355 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:38.935862 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 522/1032] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 18 08:26:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:38.941432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18feb7a-3014-407e-be7e-e925abb6e33e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:39.005955 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:39.054832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18feb7a-3014-407e-be7e-e925abb6e33e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:39.091573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de1f07-1cfe-4705-bfd9-76a662816b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:39.092275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de1f07-1cfe-4705-bfd9-76a662816b97 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:39.092555 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de1f07-1cfe-4705-bfd9-76a662816b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:39.092822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de1f07-1cfe-4705-bfd9-76a662816b97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:39.170814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de1f07-1cfe-4705-bfd9-76a662816b97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:39.172193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de1f07-1cfe-4705-bfd9-76a662816b97 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:39.233510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:39.286909 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 511/1033] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:26:38 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 08:26:39.292447 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce42859-8f8d-4ca8-aa18-4b88f168a355 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:39.406565 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:39.412166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce42859-8f8d-4ca8-aa18-4b88f168a355 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:39.452815 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad2994-99de-41aa-87a1-8cc648ba9c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:39.453647 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad2994-99de-41aa-87a1-8cc648ba9c16 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:39.453927 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad2994-99de-41aa-87a1-8cc648ba9c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:39.454199 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad2994-99de-41aa-87a1-8cc648ba9c16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:39.467050 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad2994-99de-41aa-87a1-8cc648ba9c16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:39.468619 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad2994-99de-41aa-87a1-8cc648ba9c16 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:39.491168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-cdad2994-99de-41aa-87a1-8cc648ba9c16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:39.491168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:39.491168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:39.491168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:39.491168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:39.491168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:39.491168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:39.491168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:39.491168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:39.491168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:39.491168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:39.491168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:39.491168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:39.491168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:39.491168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:39.493053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:39.493053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:39.493053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:39.493053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:39.493053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:39.493053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:39.493053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:39.493053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:39.493053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:39.493053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:39.493053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:39.493053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:39.493053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:39.493053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:39.493053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:39.493053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:39.493053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:39.495352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:39.495352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:39.495352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:39.495352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:39.499713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 523/1034] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:39.499881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de1f07-1cfe-4705-bfd9-76a662816b97 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:39.664636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de1f07-1cfe-4705-bfd9-76a662816b97 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:39.724344 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc9eff-f7db-4f38-9eb3-f424dafcee0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:39.725416 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc9eff-f7db-4f38-9eb3-f424dafcee0f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:39.725914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc9eff-f7db-4f38-9eb3-f424dafcee0f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:39.726411 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc9eff-f7db-4f38-9eb3-f424dafcee0f None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:39.727899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc9eff-f7db-4f38-9eb3-f424dafcee0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:39.728455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc9eff-f7db-4f38-9eb3-f424dafcee0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:39.731196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc9eff-f7db-4f38-9eb3-f424dafcee0f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:39.732273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa2ef7-ba0e-4e90-8194-e6976bf019f4 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:39.732575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa2ef7-ba0e-4e90-8194-e6976bf019f4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:39.732862 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa2ef7-ba0e-4e90-8194-e6976bf019f4 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:39.734543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 524/1035] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:39.741022 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a898c-f1f3-4020-aba3-81ec14521061 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:39.741330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a898c-f1f3-4020-aba3-81ec14521061 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:39.741600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a898c-f1f3-4020-aba3-81ec14521061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:39.759364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 512/1036] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:39.761082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc9eff-f7db-4f38-9eb3-f424dafcee0f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:39.761193 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-895a898c-f1f3-4020-aba3-81ec14521061 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:39.762963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-895a898c-f1f3-4020-aba3-81ec14521061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:39.792145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:39.857666 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 525/1037] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:39.885748 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a898c-f1f3-4020-aba3-81ec14521061 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:39.902166 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc9eff-f7db-4f38-9eb3-f424dafcee0f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:39.944376 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22357a0-0cf6-4049-ba6a-91c4aab69934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:39.945172 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22357a0-0cf6-4049-ba6a-91c4aab69934 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:39.945451 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22357a0-0cf6-4049-ba6a-91c4aab69934 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:39.945727 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22357a0-0cf6-4049-ba6a-91c4aab69934 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:39.952674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22357a0-0cf6-4049-ba6a-91c4aab69934 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:39.954117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22357a0-0cf6-4049-ba6a-91c4aab69934 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:39.969579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-d22357a0-0cf6-4049-ba6a-91c4aab69934 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:39.969579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:39.969579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:39.969579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:39.969579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:39.969579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:39.969579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:39.969579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:39.969579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:39.969579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:39.969579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:39.969579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:39.969579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:26:39.969579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:26:39.969579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:26:39.970553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:26:39.970553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:39.970553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:39.970553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:39.970553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:39.970553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:39.970553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:39.970553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:39.970553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:39.970553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:39.970553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:39.970553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:39.970553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:39.970553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:39.970553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:39.970553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:39.970553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:39.972251 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:26:39.972251 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:26:39.972251 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:26:39.972251 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:26:39.972251 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:26:39.972251 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:26:39.972251 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:39.972251 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:39.976547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 513/1038] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:26:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:39.977377 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22357a0-0cf6-4049-ba6a-91c4aab69934 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:39.980659 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:40.000860 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a898c-f1f3-4020-aba3-81ec14521061 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:40.039951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d56e4-82d8-434c-b3ad-40c3fbd97835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:40.040758 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d56e4-82d8-434c-b3ad-40c3fbd97835 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:40.041042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d56e4-82d8-434c-b3ad-40c3fbd97835 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:40.041318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d56e4-82d8-434c-b3ad-40c3fbd97835 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:40.052838 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d56e4-82d8-434c-b3ad-40c3fbd97835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:26:40.060544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-767d56e4-82d8-434c-b3ad-40c3fbd97835 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:26:40.060544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:40.060544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:40.060544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:40.060544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:40.060544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:40.060544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:40.060544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:40.060544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:40.060544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:40.060544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:40.060544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:40.060544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:26:40.060544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:26:40.060544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:26:40.060544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:26:40.061550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:26:40.061550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:26:40.061550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:26:40.061550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:26:40.061550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:40.061550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:40.061550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:40.061550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:40.061550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:40.061550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:40.061550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:40.061550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:40.061550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:40.061550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:40.061550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:40.061550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:40.062413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:40.062413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:40.062413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:40.062413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:40.062413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:26:40.062413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:26:40.062413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:26:40.062413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:26:40.062413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:26:40.062413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:26:40.062413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:26:40.062413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:40.062413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 526/1039] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:26:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:40.068823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d56e4-82d8-434c-b3ad-40c3fbd97835 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:40.071124 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:40.100814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22357a0-0cf6-4049-ba6a-91c4aab69934 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:40.141653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1352c1d-0c8c-4ff0-af21-e5515d742d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:40.142320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1352c1d-0c8c-4ff0-af21-e5515d742d25 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:40.142964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1352c1d-0c8c-4ff0-af21-e5515d742d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:40.142964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1352c1d-0c8c-4ff0-af21-e5515d742d25 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:40.143961 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1352c1d-0c8c-4ff0-af21-e5515d742d25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:40.144277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1352c1d-0c8c-4ff0-af21-e5515d742d25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:40.145716 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1352c1d-0c8c-4ff0-af21-e5515d742d25 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:40.159360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 514/1040] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:26:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:40.165874 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1352c1d-0c8c-4ff0-af21-e5515d742d25 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:40.184196 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d56e4-82d8-434c-b3ad-40c3fbd97835 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:40.220780 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53df848-d93f-499a-99fb-604029ca8cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:40.222064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53df848-d93f-499a-99fb-604029ca8cac None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:40.222350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53df848-d93f-499a-99fb-604029ca8cac None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:40.222626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53df848-d93f-499a-99fb-604029ca8cac None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:40.223386 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53df848-d93f-499a-99fb-604029ca8cac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:40.223719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53df848-d93f-499a-99fb-604029ca8cac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:40.225121 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53df848-d93f-499a-99fb-604029ca8cac None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:40.234436 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 527/1041] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:26:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:40.240880 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53df848-d93f-499a-99fb-604029ca8cac None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:40.243769 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:40.286573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1352c1d-0c8c-4ff0-af21-e5515d742d25 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:40.324740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0fa3ad-0ec4-493f-ad86-285b27b4c7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:40.325445 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fa3ad-0ec4-493f-ad86-285b27b4c7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:40.325731 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fa3ad-0ec4-493f-ad86-285b27b4c7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:40.326006 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fa3ad-0ec4-493f-ad86-285b27b4c7f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:40.326845 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0fa3ad-0ec4-493f-ad86-285b27b4c7f1 None admin] RBAC: Adding query filter params (role.id=9fd88609ddd942d5a9b4bc684e5e49b2, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=84cba474fc7c470ea119de697b9d7ef9) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:40.327154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0fa3ad-0ec4-493f-ad86-285b27b4c7f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:40.328637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0fa3ad-0ec4-493f-ad86-285b27b4c7f1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:40.332534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:40.348043 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 515/1042] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:26:40 2022] GET /identity/v3/role_assignments?user.id=84cba474fc7c470ea119de697b9d7ef9&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=9fd88609ddd942d5a9b4bc684e5e49b2 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:40.358926 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53df848-d93f-499a-99fb-604029ca8cac None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:40.399424 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142e96d-e83c-459e-ba80-42c61c0731de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:40.400360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142e96d-e83c-459e-ba80-42c61c0731de None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:40.400648 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142e96d-e83c-459e-ba80-42c61c0731de None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:40.400950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142e96d-e83c-459e-ba80-42c61c0731de None admin] PATH_INFO: `/v3/users/e244ec39651949f99bae847cdef6812f` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:40.417799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142e96d-e83c-459e-ba80-42c61c0731de None admin] RBAC: Authorizing `identity:get_user(user_id=e244ec39651949f99bae847cdef6812f)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:40.419315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142e96d-e83c-459e-ba80-42c61c0731de None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:40.422075 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 528/1043] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:26:40 2022] GET /identity/v3/users/e244ec39651949f99bae847cdef6812f => generated 285 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:40.428750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0fa3ad-0ec4-493f-ad86-285b27b4c7f1 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:40.541796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0fa3ad-0ec4-493f-ad86-285b27b4c7f1 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:40.581376 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d6982-9e91-4519-acb2-d4747cd0cd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:40.582082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d6982-9e91-4519-acb2-d4747cd0cd9e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:40.582374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d6982-9e91-4519-acb2-d4747cd0cd9e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:40.583581 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d6982-9e91-4519-acb2-d4747cd0cd9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:40.658283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:40.661883 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d6982-9e91-4519-acb2-d4747cd0cd9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:40.663354 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d6982-9e91-4519-acb2-d4747cd0cd9e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:40.782646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 516/1044] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:26:40 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 08:26:40.789750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142e96d-e83c-459e-ba80-42c61c0731de None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:40.868228 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:40.946095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ed84e-8b6d-453e-aed2-f6afc1e68cff None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:40.946579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142e96d-e83c-459e-ba80-42c61c0731de None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:40.947088 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ed84e-8b6d-453e-aed2-f6afc1e68cff None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:40.947088 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ed84e-8b6d-453e-aed2-f6afc1e68cff None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:40.949503 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 517/1045] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:40.955244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a3680-932c-4410-a41e-9acd86f91a34 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:40.955564 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a3680-932c-4410-a41e-9acd86f91a34 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:40.955871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a3680-932c-4410-a41e-9acd86f91a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:40.974088 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-282a3680-932c-4410-a41e-9acd86f91a34 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:40.975319 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-282a3680-932c-4410-a41e-9acd86f91a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:41.008114 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aed6df-080e-445a-bd57-e4057f7a2784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:41.009014 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aed6df-080e-445a-bd57-e4057f7a2784 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:41.009355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aed6df-080e-445a-bd57-e4057f7a2784 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:41.009630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aed6df-080e-445a-bd57-e4057f7a2784 None admin] PATH_INFO: `/v3/projects/2a3be74e0193419cb22409cb92e171e0` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:41.010853 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aed6df-080e-445a-bd57-e4057f7a2784 None admin] RBAC: Authorizing `identity:get_project(project_id=2a3be74e0193419cb22409cb92e171e0)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:41.012340 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aed6df-080e-445a-bd57-e4057f7a2784 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:41.014337 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 529/1046] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:26:40 2022] GET /identity/v3/projects/2a3be74e0193419cb22409cb92e171e0 => generated 311 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:41.020588 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aed6df-080e-445a-bd57-e4057f7a2784 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:41.071724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 518/1047] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:41.106194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a3680-932c-4410-a41e-9acd86f91a34 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:41.156465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aed6df-080e-445a-bd57-e4057f7a2784 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:41.161064 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:41.189476 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:41.198984 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef3b8b-a29e-47cd-b352-ef8dfcbb9ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:41.199653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef3b8b-a29e-47cd-b352-ef8dfcbb9ffe None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:41.199961 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef3b8b-a29e-47cd-b352-ef8dfcbb9ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:41.200241 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef3b8b-a29e-47cd-b352-ef8dfcbb9ffe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:41.201106 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef3b8b-a29e-47cd-b352-ef8dfcbb9ffe None admin] RBAC: Adding query filter params (role.id=467d49b8a0d3435aaba58464cc8ef85c, scope.project.id=2a3be74e0193419cb22409cb92e171e0, user.id=e244ec39651949f99bae847cdef6812f) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:41.201424 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef3b8b-a29e-47cd-b352-ef8dfcbb9ffe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:41.202981 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef3b8b-a29e-47cd-b352-ef8dfcbb9ffe None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:41.210225 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 530/1048] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:26:41 2022] GET /identity/v3/role_assignments?user.id=e244ec39651949f99bae847cdef6812f&scope.project.id=2a3be74e0193419cb22409cb92e171e0&role.id=467d49b8a0d3435aaba58464cc8ef85c => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:41.284262 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a3680-932c-4410-a41e-9acd86f91a34 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:41.339861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d5755-8d82-4712-88b3-9d04ae3334bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:41.340706 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d5755-8d82-4712-88b3-9d04ae3334bd None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:41.340992 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d5755-8d82-4712-88b3-9d04ae3334bd None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:41.341274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d5755-8d82-4712-88b3-9d04ae3334bd None admin] PATH_INFO: `/v3/services/image` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:41.341956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d5755-8d82-4712-88b3-9d04ae3334bd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:41.343462 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d5755-8d82-4712-88b3-9d04ae3334bd None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:41.357868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-124d5755-8d82-4712-88b3-9d04ae3334bd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 08:26:41.357868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:41.357868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:26:41.357868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:26:41.357868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:26:41.357868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:26:41.357868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:26:41.357868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:41.357868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 08:26:41.357868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:41.357868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:41.357868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:41.357868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:41.357868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:41.357868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:41.357868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:41.357868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:41.358990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:41.358990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:41.358990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:41.358990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:41.358990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:41.358990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:41.358990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:26:41.358990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:26:41.358990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:26:41.358990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:26:41.358990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:41.358990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:41.358990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:41.358990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:41.358990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:41.358990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:41.366603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:41.366603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:41.366603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:41.366603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:41.366603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:41.366603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:41.366603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:41.366603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:41.366603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:41.366603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:41.366603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:26:41.366603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:41.366603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 08:26:41.366603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:41.366603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 519/1049] 10.222.0.57 () {56 vars in 1105 bytes} [Mon Jul 18 08:26:41 2022] GET /identity/v3/services/image => generated 86 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:41.367854 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef3b8b-a29e-47cd-b352-ef8dfcbb9ffe None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:41.499603 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef3b8b-a29e-47cd-b352-ef8dfcbb9ffe None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:41.545816 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f990e-9c3b-4355-80f7-5cc949a116f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:41.546519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f990e-9c3b-4355-80f7-5cc949a116f0 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:41.546808 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f990e-9c3b-4355-80f7-5cc949a116f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:41.547072 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f990e-9c3b-4355-80f7-5cc949a116f0 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:41.547844 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f990e-9c3b-4355-80f7-5cc949a116f0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:41.548157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f990e-9c3b-4355-80f7-5cc949a116f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:41.550119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f990e-9c3b-4355-80f7-5cc949a116f0 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:41.556370 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 531/1050] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:41.562139 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d5755-8d82-4712-88b3-9d04ae3334bd None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:41.676250 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:41.691208 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d5755-8d82-4712-88b3-9d04ae3334bd None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:41.753143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd0e42-6695-4bd5-854c-d5a1bc135039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:41.754270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd0e42-6695-4bd5-854c-d5a1bc135039 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:41.754771 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd0e42-6695-4bd5-854c-d5a1bc135039 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:41.755258 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd0e42-6695-4bd5-854c-d5a1bc135039 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:41.756480 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd0e42-6695-4bd5-854c-d5a1bc135039 None admin] RBAC: Adding query filter params (type=image) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:41.757105 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd0e42-6695-4bd5-854c-d5a1bc135039 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:41.759610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd0e42-6695-4bd5-854c-d5a1bc135039 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:41.768087 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 520/1051] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:41.776213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f990e-9c3b-4355-80f7-5cc949a116f0 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:41.882152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:41.967538 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f990e-9c3b-4355-80f7-5cc949a116f0 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:42.028430 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e18db-d359-4f26-94e9-de0a5a48e2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:42.029578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e18db-d359-4f26-94e9-de0a5a48e2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:42.030088 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e18db-d359-4f26-94e9-de0a5a48e2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:42.030592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e18db-d359-4f26-94e9-de0a5a48e2a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:42.032233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e18db-d359-4f26-94e9-de0a5a48e2a1 None admin] RBAC: Adding query filter params (interface=public, service_id=5d1019a190654d0a95cc67b7e06c595a, region_id=RegionOne) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:42.032798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e18db-d359-4f26-94e9-de0a5a48e2a1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:42.035741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e18db-d359-4f26-94e9-de0a5a48e2a1 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:42.048746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 532/1052] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 18 08:26:41 2022] GET /identity/v3/endpoints?service_id=5d1019a190654d0a95cc67b7e06c595a&interface=public®ion_id=RegionOne => generated 194 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:42.051855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd0e42-6695-4bd5-854c-d5a1bc135039 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:42.209213 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd0e42-6695-4bd5-854c-d5a1bc135039 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:42.261388 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74734734-4372-47fd-9f04-5844ceccc7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:42.262464 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74734734-4372-47fd-9f04-5844ceccc7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:42.262969 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74734734-4372-47fd-9f04-5844ceccc7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:42.263476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74734734-4372-47fd-9f04-5844ceccc7f4 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:42.264831 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74734734-4372-47fd-9f04-5844ceccc7f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:42.266385 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74734734-4372-47fd-9f04-5844ceccc7f4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:42.274776 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 521/1053] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 18 08:26:42 2022] GET /identity/v3/services => generated 1957 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:26:42.470974 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d59b7-d990-42f3-8833-d2921e633540 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:42.471951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d59b7-d990-42f3-8833-d2921e633540 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:42.472159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d59b7-d990-42f3-8833-d2921e633540 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:42.474299 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 533/1054] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:42.479745 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad349874-8e54-41a0-9c97-d309efaa775f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:42.480093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad349874-8e54-41a0-9c97-d309efaa775f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:42.480316 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad349874-8e54-41a0-9c97-d309efaa775f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:42.501487 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-ad349874-8e54-41a0-9c97-d309efaa775f None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:42.503779 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ad349874-8e54-41a0-9c97-d309efaa775f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:42.528922 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:42.607556 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 522/1055] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:42.631724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d59b7-d990-42f3-8833-d2921e633540 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:42.772190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d59b7-d990-42f3-8833-d2921e633540 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:42.805226 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:42.826541 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6d8c6-abab-478d-8576-db2932f429ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:42.827454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6d8c6-abab-478d-8576-db2932f429ed None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:42.827869 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6d8c6-abab-478d-8576-db2932f429ed None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:42.828148 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6d8c6-abab-478d-8576-db2932f429ed None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:42.853653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6d8c6-abab-478d-8576-db2932f429ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:26:42.863186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-53a6d8c6-abab-478d-8576-db2932f429ed None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 08:26:42.863186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:42.863186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:42.863186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:42.863186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:42.863186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:42.863186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:42.863186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:42.863186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:42.863186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:42.863186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:42.863186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:42.863186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:26:42.863186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:26:42.863186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:26:42.863186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:26:42.865076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:26:42.865076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:26:42.865076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:26:42.865076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:26:42.865076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:42.865076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:42.865076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:42.865076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:42.865076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:42.865076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:42.865076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:42.865076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:42.865076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:42.865076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:42.865076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:42.865076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:42.866917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:42.866917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:42.866917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:42.866917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:42.866917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:26:42.866917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:26:42.866917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:26:42.866917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:26:42.866917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:26:42.866917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:26:42.866917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 08:26:42.866917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:42.874919 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 534/1056] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Jul 18 08:26:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:42.875019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad349874-8e54-41a0-9c97-d309efaa775f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:43.018846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad349874-8e54-41a0-9c97-d309efaa775f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:43.064732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63862c19-5487-4a65-a064-9b1654152c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:43.065369 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63862c19-5487-4a65-a064-9b1654152c4f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:43.065654 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63862c19-5487-4a65-a064-9b1654152c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:43.065934 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63862c19-5487-4a65-a064-9b1654152c4f None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:43.066708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63862c19-5487-4a65-a064-9b1654152c4f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:43.067150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63862c19-5487-4a65-a064-9b1654152c4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:43.068911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63862c19-5487-4a65-a064-9b1654152c4f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:43.081217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 523/1057] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 18 08:26:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:43.088075 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6d8c6-abab-478d-8576-db2932f429ed None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:43.242736 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6d8c6-abab-478d-8576-db2932f429ed None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:43.305997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1b338-4d0e-4ebe-91a7-272d1bb5954a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:43.307369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1b338-4d0e-4ebe-91a7-272d1bb5954a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:43.307904 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1b338-4d0e-4ebe-91a7-272d1bb5954a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:43.308403 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1b338-4d0e-4ebe-91a7-272d1bb5954a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:43.318493 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fc3bd-631b-4ef9-844f-f9f6a9be8539 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:43.319047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fc3bd-631b-4ef9-844f-f9f6a9be8539 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:43.319047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fc3bd-631b-4ef9-844f-f9f6a9be8539 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:43.320563 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 524/1058] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:26:43.326049 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770ddfd-db46-4b9d-b1d7-f829751c2354 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:43.326314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770ddfd-db46-4b9d-b1d7-f829751c2354 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:43.326612 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770ddfd-db46-4b9d-b1d7-f829751c2354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:43.329628 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1b338-4d0e-4ebe-91a7-272d1bb5954a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:43.332537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1b338-4d0e-4ebe-91a7-272d1bb5954a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:43.348622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-8770ddfd-db46-4b9d-b1d7-f829751c2354 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:43.349845 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8770ddfd-db46-4b9d-b1d7-f829751c2354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:43.355499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-7ce1b338-4d0e-4ebe-91a7-272d1bb5954a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:43.355499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:43.355499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:43.355499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:43.355499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:43.355499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:43.355499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:43.355499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:43.355499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:43.355499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:43.355499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:43.355499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:43.355499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:43.355499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:43.355499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:43.364739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:43.364739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:43.364739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:43.364739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:43.364739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:43.364739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:43.364739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:43.364739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:43.364739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:43.364739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:43.364739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:43.364739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:43.364739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:43.364739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:43.364739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:43.364739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:43.364739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:43.366805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:43.366805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:43.366805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:43.366805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:43.366805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 535/1059] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:43.371367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1b338-4d0e-4ebe-91a7-272d1bb5954a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:43.371555 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:43.411885 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:43.456272 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 525/1060] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:43.481412 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770ddfd-db46-4b9d-b1d7-f829751c2354 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:43.512280 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1b338-4d0e-4ebe-91a7-272d1bb5954a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:43.564654 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44a9d4-3b55-4a6f-a304-d4c9300464e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:43.566294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44a9d4-3b55-4a6f-a304-d4c9300464e2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:43.566941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44a9d4-3b55-4a6f-a304-d4c9300464e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:43.567487 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44a9d4-3b55-4a6f-a304-d4c9300464e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:43.569236 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44a9d4-3b55-4a6f-a304-d4c9300464e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:43.570074 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44a9d4-3b55-4a6f-a304-d4c9300464e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:43.572963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44a9d4-3b55-4a6f-a304-d4c9300464e2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:43.595465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 536/1061] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:43.602195 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44a9d4-3b55-4a6f-a304-d4c9300464e2 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:43.631943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770ddfd-db46-4b9d-b1d7-f829751c2354 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:43.670752 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5535a4d-5595-4300-8d98-6a3c6efe47f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:43.671572 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5535a4d-5595-4300-8d98-6a3c6efe47f2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:43.671897 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5535a4d-5595-4300-8d98-6a3c6efe47f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:43.672153 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5535a4d-5595-4300-8d98-6a3c6efe47f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:43.683145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5535a4d-5595-4300-8d98-6a3c6efe47f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:26:43.691182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-b5535a4d-5595-4300-8d98-6a3c6efe47f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:26:43.691182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:43.691182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:43.691182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:43.691182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:43.691182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:43.691182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:43.691182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:43.691182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:43.691182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:43.691182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:43.691182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:43.691182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:26:43.691182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:26:43.691182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:26:43.691182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:26:43.693402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:26:43.693402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:26:43.693402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:26:43.693402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:26:43.693402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:43.693402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:43.693402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:43.693402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:43.693402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:43.693402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:43.693402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:43.693402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:43.693402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:43.693402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:43.693402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:43.693402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:43.695534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:43.695534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:43.695534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:43.695534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:43.695534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:26:43.695534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:26:43.695534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:26:43.695534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:26:43.695534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:26:43.695534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:26:43.695534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:26:43.695534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:43.695534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 526/1062] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:26:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:43.698588 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5535a4d-5595-4300-8d98-6a3c6efe47f2 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:43.702503 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:43.760182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:43.774421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44a9d4-3b55-4a6f-a304-d4c9300464e2 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:43.828182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcd8db-654d-4632-8235-dfa754f92e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:43.828866 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcd8db-654d-4632-8235-dfa754f92e64 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:43.829153 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcd8db-654d-4632-8235-dfa754f92e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:43.829431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcd8db-654d-4632-8235-dfa754f92e64 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:43.838912 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcd8db-654d-4632-8235-dfa754f92e64 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:43.840435 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcd8db-654d-4632-8235-dfa754f92e64 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:43.854836 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5535a4d-5595-4300-8d98-6a3c6efe47f2 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:43.868762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-98bcd8db-654d-4632-8235-dfa754f92e64 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 08:26:43.868762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:43.868762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:43.868762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:43.868762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:43.868762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:43.868762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:43.868762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:43.868762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:43.868762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:43.868762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:43.868762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:43.868762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:26:43.868762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:26:43.868762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:26:43.868762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:26:43.870210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:43.870210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:43.870210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:26:43.870210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:43.870210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:26:43.870210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:43.870210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:43.870210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:43.870210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:43.870210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:43.870210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:43.870210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:43.870210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:43.870210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:43.870210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:43.870210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:43.870210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:43.870210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:43.871609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:43.871609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:43.871609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:26:43.871609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:26:43.871609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:26:43.871609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:26:43.871609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:26:43.871609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:26:43.871609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 08:26:43.871609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:43.871609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 537/1063] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 18 08:26:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:43.878273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcd8db-654d-4632-8235-dfa754f92e64 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:43.923790 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b36e0-0779-4311-bc33-e637ab534221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:43.925450 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b36e0-0779-4311-bc33-e637ab534221 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:43.925813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b36e0-0779-4311-bc33-e637ab534221 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:43.926157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b36e0-0779-4311-bc33-e637ab534221 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:43.927040 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b36e0-0779-4311-bc33-e637ab534221 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:43.927467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b36e0-0779-4311-bc33-e637ab534221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:43.929104 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b36e0-0779-4311-bc33-e637ab534221 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:43.948694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 527/1064] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:26:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:43.951157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b36e0-0779-4311-bc33-e637ab534221 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:44.036037 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcd8db-654d-4632-8235-dfa754f92e64 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:44.042485 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:44.084731 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b36e0-0779-4311-bc33-e637ab534221 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:44.097455 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7338d-cfeb-4e03-a035-0467555d2dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:44.098858 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7338d-cfeb-4e03-a035-0467555d2dda None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:44.099347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7338d-cfeb-4e03-a035-0467555d2dda None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:44.099870 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7338d-cfeb-4e03-a035-0467555d2dda None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:44.101504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7338d-cfeb-4e03-a035-0467555d2dda None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:44.102060 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7338d-cfeb-4e03-a035-0467555d2dda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:44.104706 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7338d-cfeb-4e03-a035-0467555d2dda None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:44.125702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ae663-6c45-4539-a0c9-3e4256ce043a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:44.126604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ae663-6c45-4539-a0c9-3e4256ce043a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:44.126880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ae663-6c45-4539-a0c9-3e4256ce043a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:44.127154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ae663-6c45-4539-a0c9-3e4256ce043a None admin] PATH_INFO: `/v3/users/e244ec39651949f99bae847cdef6812f` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:44.129117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ae663-6c45-4539-a0c9-3e4256ce043a None admin] RBAC: Authorizing `identity:get_user(user_id=e244ec39651949f99bae847cdef6812f)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:44.130645 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ae663-6c45-4539-a0c9-3e4256ce043a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:44.132649 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 528/1065] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:26:43 2022] GET /identity/v3/users/e244ec39651949f99bae847cdef6812f => generated 285 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:44.138361 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ae663-6c45-4539-a0c9-3e4256ce043a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:44.139702 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 538/1066] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 18 08:26:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:44.149883 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7338d-cfeb-4e03-a035-0467555d2dda None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:44.171103 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:44.264240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ae663-6c45-4539-a0c9-3e4256ce043a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:44.301390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a747d-2e2c-4986-ae24-ff331c02e5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:44.302562 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a747d-2e2c-4986-ae24-ff331c02e5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:44.302847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a747d-2e2c-4986-ae24-ff331c02e5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:44.303121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a747d-2e2c-4986-ae24-ff331c02e5f8 None admin] PATH_INFO: `/v3/projects/2a3be74e0193419cb22409cb92e171e0` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:44.304409 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a747d-2e2c-4986-ae24-ff331c02e5f8 None admin] RBAC: Authorizing `identity:get_project(project_id=2a3be74e0193419cb22409cb92e171e0)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:44.305868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a747d-2e2c-4986-ae24-ff331c02e5f8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:44.312709 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 529/1067] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:26:44 2022] GET /identity/v3/projects/2a3be74e0193419cb22409cb92e171e0 => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:44.313912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a747d-2e2c-4986-ae24-ff331c02e5f8 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:44.317841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:44.329328 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7338d-cfeb-4e03-a035-0467555d2dda None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:44.386280 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc17836-3cd4-4aaf-9bb9-3701ffb71db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:44.387484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc17836-3cd4-4aaf-9bb9-3701ffb71db5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:44.388008 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc17836-3cd4-4aaf-9bb9-3701ffb71db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:44.388471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc17836-3cd4-4aaf-9bb9-3701ffb71db5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:44.407975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc17836-3cd4-4aaf-9bb9-3701ffb71db5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:44.410600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc17836-3cd4-4aaf-9bb9-3701ffb71db5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:44.442724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-0dc17836-3cd4-4aaf-9bb9-3701ffb71db5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:44.442724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:44.442724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:44.442724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:44.442724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:44.442724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:44.442724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:44.442724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:44.442724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:44.442724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:44.442724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:44.442724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:44.442724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:44.442724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:44.442724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:44.443986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:44.443986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:44.443986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:44.443986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:44.443986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:44.443986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:44.443986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:44.443986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:44.443986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:44.443986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:44.443986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:44.443986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:44.443986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:44.443986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:44.443986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:44.443986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:44.443986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:44.445177 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:44.445177 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:44.445177 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:44.445177 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:44.448089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 539/1068] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:44.453988 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc17836-3cd4-4aaf-9bb9-3701ffb71db5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:44.479036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a747d-2e2c-4986-ae24-ff331c02e5f8 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:44.527175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb0b82-c43f-46af-8baf-0f54e7ce0ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:44.528267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb0b82-c43f-46af-8baf-0f54e7ce0ade None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:44.528644 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb0b82-c43f-46af-8baf-0f54e7ce0ade None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:44.529106 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb0b82-c43f-46af-8baf-0f54e7ce0ade None admin] PATH_INFO: `/v3/projects/2a3be74e0193419cb22409cb92e171e0/users/e244ec39651949f99bae847cdef6812f/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:44.531614 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb0b82-c43f-46af-8baf-0f54e7ce0ade None admin] RBAC: Authorizing `identity:create_grant(project_id=2a3be74e0193419cb22409cb92e171e0, user_id=e244ec39651949f99bae847cdef6812f, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:44.533517 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb0b82-c43f-46af-8baf-0f54e7ce0ade None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:44.538700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:44.580084 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc17836-3cd4-4aaf-9bb9-3701ffb71db5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:44.609399 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 530/1069] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:26:44 2022] PUT /identity/v3/projects/2a3be74e0193419cb22409cb92e171e0/users/e244ec39651949f99bae847cdef6812f/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:26:44.613689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec893f4-1a73-4884-b1ce-e8127bbf03f8 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:44.614253 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec893f4-1a73-4884-b1ce-e8127bbf03f8 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:44.614790 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec893f4-1a73-4884-b1ce-e8127bbf03f8 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:44.619744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 531/1070] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:44.623242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3396f-d4a3-4a7e-925f-511bc7d06e24 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:44.623539 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3396f-d4a3-4a7e-925f-511bc7d06e24 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:44.623860 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3396f-d4a3-4a7e-925f-511bc7d06e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:44.626045 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de42bc-1efa-4771-bd73-0d52d3098874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:44.627446 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de42bc-1efa-4771-bd73-0d52d3098874 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:44.628527 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de42bc-1efa-4771-bd73-0d52d3098874 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:44.629099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de42bc-1efa-4771-bd73-0d52d3098874 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:44.631843 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de42bc-1efa-4771-bd73-0d52d3098874 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:44.632202 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de42bc-1efa-4771-bd73-0d52d3098874 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:44.633695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de42bc-1efa-4771-bd73-0d52d3098874 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:44.647412 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-68c3396f-d4a3-4a7e-925f-511bc7d06e24 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:44.648591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-68c3396f-d4a3-4a7e-925f-511bc7d06e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:44.651348 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 540/1071] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:44.657458 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de42bc-1efa-4771-bd73-0d52d3098874 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:44.751167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:44.765636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 532/1072] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:44.790519 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3396f-d4a3-4a7e-925f-511bc7d06e24 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:44.830720 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de42bc-1efa-4771-bd73-0d52d3098874 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:44.870117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:44.883294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49a5c3-c477-45fd-b8d3-bf9e92281e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:44.884418 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49a5c3-c477-45fd-b8d3-bf9e92281e6c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:44.884802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49a5c3-c477-45fd-b8d3-bf9e92281e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:44.885544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49a5c3-c477-45fd-b8d3-bf9e92281e6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:44.893790 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49a5c3-c477-45fd-b8d3-bf9e92281e6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:44.896034 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49a5c3-c477-45fd-b8d3-bf9e92281e6c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:44.912545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-2e49a5c3-c477-45fd-b8d3-bf9e92281e6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:44.912545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:44.912545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:44.912545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:44.912545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:44.912545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:44.912545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:44.912545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:44.912545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:44.912545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:44.912545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:44.912545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:44.912545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:26:44.912545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:26:44.912545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:26:44.913530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:26:44.913530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:44.913530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:44.913530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:44.913530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:44.913530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:44.913530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:44.913530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:44.913530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:44.913530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:44.913530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:44.913530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:44.913530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:44.913530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:44.913530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:44.913530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:44.913530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:44.914616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:26:44.914616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:26:44.914616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:26:44.914616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:26:44.914616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:26:44.914616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:26:44.914616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:44.914616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:44.914616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 541/1073] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:26:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:44.921301 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49a5c3-c477-45fd-b8d3-bf9e92281e6c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:44.959965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3396f-d4a3-4a7e-925f-511bc7d06e24 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:45.005396 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee929e8-742c-4c10-977b-25f0f4a95ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:45.006176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee929e8-742c-4c10-977b-25f0f4a95ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:45.006457 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee929e8-742c-4c10-977b-25f0f4a95ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:45.006715 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee929e8-742c-4c10-977b-25f0f4a95ac0 None admin] PATH_INFO: `/v3/services/image` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:45.007382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee929e8-742c-4c10-977b-25f0f4a95ac0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:45.008807 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee929e8-742c-4c10-977b-25f0f4a95ac0 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:45.022370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-5ee929e8-742c-4c10-977b-25f0f4a95ac0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 08:26:45.022370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:45.022370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:26:45.022370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:26:45.022370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:26:45.022370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:26:45.022370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:26:45.022370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:45.022370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 08:26:45.022370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:45.022370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:26:45.022370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:45.022370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:45.022370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:45.022370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:45.022370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:45.022370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:45.024492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:45.024492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:45.024492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:45.024492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:45.024492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:45.024492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:45.024492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:26:45.024492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:26:45.024492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:26:45.024492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:26:45.024492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:45.024492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:45.024492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:45.024492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:45.024492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:45.024492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:45.026117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:45.026117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:45.026117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:45.026117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:45.026117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:45.026117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:45.026117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:45.026117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:45.026117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:45.026117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:45.026117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:26:45.026117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:26:45.026117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 08:26:45.026117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:45.026117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 533/1074] 10.222.0.57 () {56 vars in 1105 bytes} [Mon Jul 18 08:26:44 2022] GET /identity/v3/services/image => generated 86 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:45.029991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee929e8-742c-4c10-977b-25f0f4a95ac0 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:45.070013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49a5c3-c477-45fd-b8d3-bf9e92281e6c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:45.102320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:45.120419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cb47f-54c8-43b2-b6d9-4d142f54d3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:45.125859 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cb47f-54c8-43b2-b6d9-4d142f54d3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:45.125859 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cb47f-54c8-43b2-b6d9-4d142f54d3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:45.125859 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cb47f-54c8-43b2-b6d9-4d142f54d3f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:45.125859 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cb47f-54c8-43b2-b6d9-4d142f54d3f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:45.125859 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cb47f-54c8-43b2-b6d9-4d142f54d3f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:45.125859 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cb47f-54c8-43b2-b6d9-4d142f54d3f9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:45.139379 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 542/1075] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:26:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:45.145756 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cb47f-54c8-43b2-b6d9-4d142f54d3f9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:45.176693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee929e8-742c-4c10-977b-25f0f4a95ac0 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:45.200606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:45.240051 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065ebd3-9921-4c2e-a6f6-57154bf9cc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:45.241089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065ebd3-9921-4c2e-a6f6-57154bf9cc3b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:45.241542 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065ebd3-9921-4c2e-a6f6-57154bf9cc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:45.241971 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065ebd3-9921-4c2e-a6f6-57154bf9cc3b None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:45.243167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d065ebd3-9921-4c2e-a6f6-57154bf9cc3b None admin] RBAC: Adding query filter params (name=image) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:45.243698 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d065ebd3-9921-4c2e-a6f6-57154bf9cc3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:45.246000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d065ebd3-9921-4c2e-a6f6-57154bf9cc3b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:45.254965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 534/1076] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:45.260729 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065ebd3-9921-4c2e-a6f6-57154bf9cc3b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:45.275058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cb47f-54c8-43b2-b6d9-4d142f54d3f9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:45.321856 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4674b06-d6cb-4946-a5ca-295857589c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:45.322748 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4674b06-d6cb-4946-a5ca-295857589c0d None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:45.323047 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4674b06-d6cb-4946-a5ca-295857589c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:45.323297 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4674b06-d6cb-4946-a5ca-295857589c0d None admin] PATH_INFO: `/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/84cba474fc7c470ea119de697b9d7ef9/roles/9fd88609ddd942d5a9b4bc684e5e49b2` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:45.325668 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4674b06-d6cb-4946-a5ca-295857589c0d None admin] RBAC: Authorizing `identity:create_grant(project_id=35c846931fd443fbb83a6fbe0aeaa866, user_id=84cba474fc7c470ea119de697b9d7ef9, role_id=9fd88609ddd942d5a9b4bc684e5e49b2)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:45.327138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4674b06-d6cb-4946-a5ca-295857589c0d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:45.376388 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065ebd3-9921-4c2e-a6f6-57154bf9cc3b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:45.400877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:45.404446 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 543/1077] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:26:45 2022] PUT /identity/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/84cba474fc7c470ea119de697b9d7ef9/roles/9fd88609ddd942d5a9b4bc684e5e49b2 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:26:45.418908 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e384d-3e09-4d2f-9dfd-0b078736dc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:45.422213 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e384d-3e09-4d2f-9dfd-0b078736dc9b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:45.422213 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e384d-3e09-4d2f-9dfd-0b078736dc9b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:45.422213 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e384d-3e09-4d2f-9dfd-0b078736dc9b None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:45.422213 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e384d-3e09-4d2f-9dfd-0b078736dc9b None admin] RBAC: Adding query filter params (type=image) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:45.422213 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e384d-3e09-4d2f-9dfd-0b078736dc9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:45.424894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e384d-3e09-4d2f-9dfd-0b078736dc9b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:45.435951 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 535/1078] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:45.438101 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4674b06-d6cb-4946-a5ca-295857589c0d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:45.568601 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4674b06-d6cb-4946-a5ca-295857589c0d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:45.570370 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:45.611894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727f538-8e09-414c-abce-e94434a6eff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:45.611894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727f538-8e09-414c-abce-e94434a6eff3 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:45.612347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727f538-8e09-414c-abce-e94434a6eff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:45.612965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727f538-8e09-414c-abce-e94434a6eff3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:45.613637 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727f538-8e09-414c-abce-e94434a6eff3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:45.615655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727f538-8e09-414c-abce-e94434a6eff3 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:45.723763 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 544/1079] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 18 08:26:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:26:46.710247 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16026a-9931-4fd9-b746-c49804c4bc8e None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:46.710820 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16026a-9931-4fd9-b746-c49804c4bc8e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:46.710820 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16026a-9931-4fd9-b746-c49804c4bc8e None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:46.712269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 536/1080] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:46.717790 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc3c53-d2af-4ed0-a15a-fbabe8704010 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:46.718366 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc3c53-d2af-4ed0-a15a-fbabe8704010 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:46.718366 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc3c53-d2af-4ed0-a15a-fbabe8704010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:46.737135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-02bc3c53-d2af-4ed0-a15a-fbabe8704010 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:46.738373 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-02bc3c53-d2af-4ed0-a15a-fbabe8704010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:46.823233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 545/1081] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:46.848217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16026a-9931-4fd9-b746-c49804c4bc8e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:46.988437 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16026a-9931-4fd9-b746-c49804c4bc8e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:47.031706 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb98ec-08ef-41c7-a31c-b08acbfb68b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:47.032498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb98ec-08ef-41c7-a31c-b08acbfb68b5 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:47.032781 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb98ec-08ef-41c7-a31c-b08acbfb68b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:47.033058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb98ec-08ef-41c7-a31c-b08acbfb68b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:47.048502 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb98ec-08ef-41c7-a31c-b08acbfb68b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:26:47.057362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-5fcb98ec-08ef-41c7-a31c-b08acbfb68b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:26:47.057362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:47.057362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:47.057362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:47.057362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:47.057362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:47.057362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:47.057362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:47.057362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:47.057362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:47.057362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:47.057362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:47.057362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:26:47.057362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:26:47.057362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:26:47.057362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:26:47.058429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:26:47.058429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:26:47.058429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:26:47.058429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:26:47.058429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:47.058429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:47.058429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:47.058429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:47.058429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:47.058429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:47.058429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:47.058429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:47.058429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:47.058429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:47.058429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:47.058429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:47.059291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:47.059291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:47.059291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:47.059291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:47.059291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:26:47.059291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:26:47.059291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:26:47.059291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:26:47.059291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:26:47.059291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:26:47.059291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:26:47.059291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:47.064198 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 537/1082] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:26:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:47.065606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc3c53-d2af-4ed0-a15a-fbabe8704010 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:47.159981 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:47.179636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc3c53-d2af-4ed0-a15a-fbabe8704010 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:47.243286 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbae358-ae7c-499c-8295-e1e9b2b6962e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:47.244402 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbae358-ae7c-499c-8295-e1e9b2b6962e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:47.244911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbae358-ae7c-499c-8295-e1e9b2b6962e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:47.245398 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbae358-ae7c-499c-8295-e1e9b2b6962e None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:47.246839 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbae358-ae7c-499c-8295-e1e9b2b6962e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:47.247395 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbae358-ae7c-499c-8295-e1e9b2b6962e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:47.249988 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbae358-ae7c-499c-8295-e1e9b2b6962e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:47.271606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 546/1083] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:26:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:47.274030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb98ec-08ef-41c7-a31c-b08acbfb68b5 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:47.336978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:47.409061 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb98ec-08ef-41c7-a31c-b08acbfb68b5 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:47.447555 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33badf78-a32b-4c9e-b18f-abe2b20d40c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:47.448733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33badf78-a32b-4c9e-b18f-abe2b20d40c6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:47.449178 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33badf78-a32b-4c9e-b18f-abe2b20d40c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:47.449597 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33badf78-a32b-4c9e-b18f-abe2b20d40c6 None admin] PATH_INFO: `/v3/users/e244ec39651949f99bae847cdef6812f` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:47.450995 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33badf78-a32b-4c9e-b18f-abe2b20d40c6 None admin] RBAC: Authorizing `identity:get_user(user_id=e244ec39651949f99bae847cdef6812f)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:47.452643 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33badf78-a32b-4c9e-b18f-abe2b20d40c6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:47.454555 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 538/1084] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:26:47 2022] GET /identity/v3/users/e244ec39651949f99bae847cdef6812f => generated 285 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:47.461413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbae358-ae7c-499c-8295-e1e9b2b6962e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:47.606341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbae358-ae7c-499c-8295-e1e9b2b6962e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:47.660821 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a4ec1-6097-4e3c-91ea-c769d4dac4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:47.661935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a4ec1-6097-4e3c-91ea-c769d4dac4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:47.662223 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a4ec1-6097-4e3c-91ea-c769d4dac4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:47.662717 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a4ec1-6097-4e3c-91ea-c769d4dac4e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:47.679246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7794e2-5236-477a-b13e-d7cc6e39adaa None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:47.679534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7794e2-5236-477a-b13e-d7cc6e39adaa None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:47.680211 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7794e2-5236-477a-b13e-d7cc6e39adaa None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:47.681598 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 539/1085] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:47.687066 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9f790-b82b-463b-8bb7-c810b9b6c332 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:47.687416 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9f790-b82b-463b-8bb7-c810b9b6c332 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:47.687632 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9f790-b82b-463b-8bb7-c810b9b6c332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:47.693200 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:47.708815 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-1df9f790-b82b-463b-8bb7-c810b9b6c332 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:47.710112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1df9f790-b82b-463b-8bb7-c810b9b6c332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:47.766090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a4ec1-6097-4e3c-91ea-c769d4dac4e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:47.769503 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a4ec1-6097-4e3c-91ea-c769d4dac4e8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:47.791227 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 540/1086] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:47.821955 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9f790-b82b-463b-8bb7-c810b9b6c332 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:47.876454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:47.923188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 547/1087] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:26:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 08:26:47.931095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a4ec1-6097-4e3c-91ea-c769d4dac4e8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:47.960574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9f790-b82b-463b-8bb7-c810b9b6c332 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:47.995370 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:48.015909 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e23968-0baf-4aca-bb8c-222dd9022d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:48.016754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e23968-0baf-4aca-bb8c-222dd9022d13 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:48.017188 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e23968-0baf-4aca-bb8c-222dd9022d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:48.017668 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e23968-0baf-4aca-bb8c-222dd9022d13 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:48.031340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e23968-0baf-4aca-bb8c-222dd9022d13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:26:48.039865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-03e23968-0baf-4aca-bb8c-222dd9022d13 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 08:26:48.039865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:48.039865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:48.039865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:48.039865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:48.039865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:48.039865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:48.039865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:48.039865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:48.039865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:48.039865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:48.039865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:48.039865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:26:48.039865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:26:48.039865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:26:48.039865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:26:48.041646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:26:48.041646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:26:48.041646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:26:48.041646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:26:48.041646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:48.041646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:48.041646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:48.041646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:48.041646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:48.041646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:48.041646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:48.041646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:48.041646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:48.041646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:48.041646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:48.041646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:48.043314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:48.043314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:48.043314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:48.043314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:48.043314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:26:48.043314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:26:48.043314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:26:48.043314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:26:48.043314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:26:48.043314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:26:48.043314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 08:26:48.043314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:48.043314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 541/1088] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Jul 18 08:26:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:48.043314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eea0c2-2698-4221-85a4-5917f045fdb7 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:48.044588 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eea0c2-2698-4221-85a4-5917f045fdb7 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:48.044588 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eea0c2-2698-4221-85a4-5917f045fdb7 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:48.047812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 542/1089] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:48.047812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eea0c2-2698-4221-85a4-5917f045fdb7 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:48.058984 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a4ec1-6097-4e3c-91ea-c769d4dac4e8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:48.099775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec2522-991a-4374-9d50-8fc22a7573cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:48.100641 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec2522-991a-4374-9d50-8fc22a7573cf None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:48.100920 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec2522-991a-4374-9d50-8fc22a7573cf None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:48.101267 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec2522-991a-4374-9d50-8fc22a7573cf None admin] PATH_INFO: `/v3/projects/2a3be74e0193419cb22409cb92e171e0` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:48.102416 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec2522-991a-4374-9d50-8fc22a7573cf None admin] RBAC: Authorizing `identity:get_project(project_id=2a3be74e0193419cb22409cb92e171e0)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:48.103898 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec2522-991a-4374-9d50-8fc22a7573cf None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:48.105635 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 548/1090] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:26:47 2022] GET /identity/v3/projects/2a3be74e0193419cb22409cb92e171e0 => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:48.106828 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7897b-ea1f-42e0-88a1-ab9d69a79cb4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:48.107147 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7897b-ea1f-42e0-88a1-ab9d69a79cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:48.107385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7897b-ea1f-42e0-88a1-ab9d69a79cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:48.127481 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-6bd7897b-ea1f-42e0-88a1-ab9d69a79cb4 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:48.129293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6bd7897b-ea1f-42e0-88a1-ab9d69a79cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:48.168963 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eea0c2-2698-4221-85a4-5917f045fdb7 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:48.171882 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:48.210726 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159260d5-74b2-4a6c-a1a1-6c08d839362b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:48.211432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159260d5-74b2-4a6c-a1a1-6c08d839362b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:48.211757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159260d5-74b2-4a6c-a1a1-6c08d839362b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:48.212104 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159260d5-74b2-4a6c-a1a1-6c08d839362b None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:48.212495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 549/1091] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:48.212889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159260d5-74b2-4a6c-a1a1-6c08d839362b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:48.213251 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159260d5-74b2-4a6c-a1a1-6c08d839362b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:48.213358 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7897b-ea1f-42e0-88a1-ab9d69a79cb4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:48.214688 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159260d5-74b2-4a6c-a1a1-6c08d839362b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:48.225812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 543/1092] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 18 08:26:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:48.231597 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159260d5-74b2-4a6c-a1a1-6c08d839362b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:48.277936 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:48.339809 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7897b-ea1f-42e0-88a1-ab9d69a79cb4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:48.353170 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159260d5-74b2-4a6c-a1a1-6c08d839362b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:48.376608 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04de0f-7cbf-48f2-b2cb-19cad4b01143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:48.377957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04de0f-7cbf-48f2-b2cb-19cad4b01143 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:48.378380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04de0f-7cbf-48f2-b2cb-19cad4b01143 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:48.378644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04de0f-7cbf-48f2-b2cb-19cad4b01143 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:48.379462 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04de0f-7cbf-48f2-b2cb-19cad4b01143 None admin] RBAC: Adding query filter params (role.id=467d49b8a0d3435aaba58464cc8ef85c, scope.project.id=2a3be74e0193419cb22409cb92e171e0, user.id=e244ec39651949f99bae847cdef6812f) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:48.379904 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04de0f-7cbf-48f2-b2cb-19cad4b01143 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:48.381277 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04de0f-7cbf-48f2-b2cb-19cad4b01143 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:48.387638 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 550/1093] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:26:48 2022] GET /identity/v3/role_assignments?user.id=e244ec39651949f99bae847cdef6812f&scope.project.id=2a3be74e0193419cb22409cb92e171e0&role.id=467d49b8a0d3435aaba58464cc8ef85c => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:48.388531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04de0f-7cbf-48f2-b2cb-19cad4b01143 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:48.396235 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617573da-f8f7-45e3-aeda-5e5a83a17380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:48.397407 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617573da-f8f7-45e3-aeda-5e5a83a17380 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:48.397694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617573da-f8f7-45e3-aeda-5e5a83a17380 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:48.397972 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617573da-f8f7-45e3-aeda-5e5a83a17380 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:48.413432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617573da-f8f7-45e3-aeda-5e5a83a17380 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:48.414955 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617573da-f8f7-45e3-aeda-5e5a83a17380 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:48.436828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-617573da-f8f7-45e3-aeda-5e5a83a17380 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:48.436828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:48.436828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:48.436828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:48.436828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:48.436828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:48.436828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:48.436828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:48.436828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:48.436828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:48.436828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:48.436828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:48.436828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:48.436828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:48.436828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:48.437938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:48.437938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:48.437938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:48.437938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:48.437938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:48.437938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:48.437938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:48.437938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:48.437938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:48.437938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:48.437938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:48.437938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:48.437938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:48.437938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:48.437938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:48.437938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:48.437938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:48.438928 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:48.438928 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:48.438928 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:48.438928 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:48.438928 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 544/1094] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:48.444281 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617573da-f8f7-45e3-aeda-5e5a83a17380 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:48.467115 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:48.529190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04de0f-7cbf-48f2-b2cb-19cad4b01143 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:48.546646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:48.574020 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617573da-f8f7-45e3-aeda-5e5a83a17380 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:48.589555 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add8b95c-10b1-47eb-9822-194238ae7eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:48.589555 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8b95c-10b1-47eb-9822-194238ae7eae None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:48.590024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8b95c-10b1-47eb-9822-194238ae7eae None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:48.590755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8b95c-10b1-47eb-9822-194238ae7eae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:48.628083 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad851d5f-5045-4c31-8e27-5fbc5980e7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:48.628670 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad851d5f-5045-4c31-8e27-5fbc5980e7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:48.629159 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad851d5f-5045-4c31-8e27-5fbc5980e7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:48.629495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad851d5f-5045-4c31-8e27-5fbc5980e7b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:48.630247 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad851d5f-5045-4c31-8e27-5fbc5980e7b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:48.630559 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad851d5f-5045-4c31-8e27-5fbc5980e7b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:48.632149 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad851d5f-5045-4c31-8e27-5fbc5980e7b9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:48.646724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 545/1095] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:48.652714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad851d5f-5045-4c31-8e27-5fbc5980e7b9 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:48.690780 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add8b95c-10b1-47eb-9822-194238ae7eae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:48.692779 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add8b95c-10b1-47eb-9822-194238ae7eae None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:48.747810 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:48.774424 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad851d5f-5045-4c31-8e27-5fbc5980e7b9 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:48.815644 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9845a-968a-4207-b8b1-38b0a475c7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:48.817872 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9845a-968a-4207-b8b1-38b0a475c7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:48.818637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9845a-968a-4207-b8b1-38b0a475c7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:48.819104 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9845a-968a-4207-b8b1-38b0a475c7c6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:48.821003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 551/1096] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:26:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 08:26:48.827893 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add8b95c-10b1-47eb-9822-194238ae7eae None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:48.835889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a9845a-968a-4207-b8b1-38b0a475c7c6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:48.835889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a9845a-968a-4207-b8b1-38b0a475c7c6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:48.839144 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:48.854620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-c4a9845a-968a-4207-b8b1-38b0a475c7c6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 08:26:48.854620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:48.854620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:48.854620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:48.854620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:48.854620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:48.854620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:48.854620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:48.854620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:48.854620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:48.854620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:48.854620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:48.854620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:26:48.854620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:26:48.854620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:26:48.854620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:26:48.855655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:48.855655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:48.855655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:26:48.855655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:48.855655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:26:48.855655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:26:48.855655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:48.855655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:48.855655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:48.855655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:48.855655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:48.855655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:48.855655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:48.855655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:48.855655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:48.855655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:48.855655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:48.855655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:48.862634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:48.862634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:48.862634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:26:48.862634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:26:48.862634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:26:48.862634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:26:48.862634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:26:48.862634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:26:48.862634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 08:26:48.862634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:26:48.862634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 546/1097] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Jul 18 08:26:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:48.862634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9845a-968a-4207-b8b1-38b0a475c7c6 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:48.947048 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add8b95c-10b1-47eb-9822-194238ae7eae None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:48.975754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9845a-968a-4207-b8b1-38b0a475c7c6 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:48.983584 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605b73c-faf3-48f1-ad9a-10d86d29dd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:48.984407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605b73c-faf3-48f1-ad9a-10d86d29dd9e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:48.984690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605b73c-faf3-48f1-ad9a-10d86d29dd9e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:48.984967 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605b73c-faf3-48f1-ad9a-10d86d29dd9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:48.996955 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605b73c-faf3-48f1-ad9a-10d86d29dd9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:48.998417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605b73c-faf3-48f1-ad9a-10d86d29dd9e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:49.012618 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a8718-83b6-4b64-a2c6-1e90d705531f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:49.013297 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a8718-83b6-4b64-a2c6-1e90d705531f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:49.013579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a8718-83b6-4b64-a2c6-1e90d705531f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:49.013855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a8718-83b6-4b64-a2c6-1e90d705531f None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:49.014700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a8718-83b6-4b64-a2c6-1e90d705531f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:49.015007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a8718-83b6-4b64-a2c6-1e90d705531f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:49.016460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a8718-83b6-4b64-a2c6-1e90d705531f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:49.017003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-5605b73c-faf3-48f1-ad9a-10d86d29dd9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:49.017003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:49.017003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:49.017003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:49.017003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:49.017003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:49.017003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:49.017003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:49.017003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:49.017003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:49.017003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:49.017003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:49.017003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:49.017003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:49.017003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:49.017903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:49.017903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:49.017903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:49.017903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:49.017903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:49.017903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:49.017903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:49.017903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:49.017903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:49.017903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:49.017903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:49.017903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:49.017903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:49.017903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:49.017903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:49.017903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:49.017903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:49.018951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:49.018951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:49.018951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:49.018951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:49.018951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 552/1098] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:26:49.021204 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:49.027486 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605b73c-faf3-48f1-ad9a-10d86d29dd9e None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:49.043147 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 547/1099] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Jul 18 08:26:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:49.044019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a8718-83b6-4b64-a2c6-1e90d705531f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:49.092673 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:49.137022 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605b73c-faf3-48f1-ad9a-10d86d29dd9e None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:49.151459 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a8718-83b6-4b64-a2c6-1e90d705531f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:49.173713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496b2d5-1a92-4fa6-b158-afda342d1999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:49.175107 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496b2d5-1a92-4fa6-b158-afda342d1999 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:49.175397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496b2d5-1a92-4fa6-b158-afda342d1999 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:49.175704 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496b2d5-1a92-4fa6-b158-afda342d1999 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:49.176450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496b2d5-1a92-4fa6-b158-afda342d1999 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:49.176760 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496b2d5-1a92-4fa6-b158-afda342d1999 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:49.178133 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496b2d5-1a92-4fa6-b158-afda342d1999 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:49.189091 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f8950b-9bb4-40f0-a93b-dc7d627d48e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:49.190094 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8950b-9bb4-40f0-a93b-dc7d627d48e5 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:49.190231 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 553/1100] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:49.190323 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8950b-9bb4-40f0-a93b-dc7d627d48e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:49.190493 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8950b-9bb4-40f0-a93b-dc7d627d48e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:49.264578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:49.273741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8950b-9bb4-40f0-a93b-dc7d627d48e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:49.275271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8950b-9bb4-40f0-a93b-dc7d627d48e5 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:49.394758 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 548/1101] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:26:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 08:26:49.398692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496b2d5-1a92-4fa6-b158-afda342d1999 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:49.509528 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496b2d5-1a92-4fa6-b158-afda342d1999 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:49.530011 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:49.544777 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd0133-57e0-4a9a-bd5e-81f946f6b216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:49.545585 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd0133-57e0-4a9a-bd5e-81f946f6b216 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:49.545873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd0133-57e0-4a9a-bd5e-81f946f6b216 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:49.546152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd0133-57e0-4a9a-bd5e-81f946f6b216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:49.557378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd0133-57e0-4a9a-bd5e-81f946f6b216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:49.558831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd0133-57e0-4a9a-bd5e-81f946f6b216 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:49.576772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-4efd0133-57e0-4a9a-bd5e-81f946f6b216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:49.576772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:49.576772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:49.576772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:49.576772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:49.576772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:49.576772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:49.576772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:49.576772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:49.576772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:49.576772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:49.576772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:49.576772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:26:49.576772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:26:49.576772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:26:49.577746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:26:49.577746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:49.577746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:49.577746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:49.577746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:49.577746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:49.577746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:49.577746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:49.577746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:49.577746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:49.577746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:49.577746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:49.577746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:49.577746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:49.577746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:49.577746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:49.577746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:49.578880 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:26:49.578880 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:26:49.578880 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:26:49.578880 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:49.578880 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 554/1102] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:49.584690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f8950b-9bb4-40f0-a93b-dc7d627d48e5 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:49.680122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:49.687122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f8950b-9bb4-40f0-a93b-dc7d627d48e5 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:49.723248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b88afc-30ce-4fa0-89f7-139f49906fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:49.723972 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b88afc-30ce-4fa0-89f7-139f49906fdb None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:49.724257 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b88afc-30ce-4fa0-89f7-139f49906fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:49.724535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b88afc-30ce-4fa0-89f7-139f49906fdb None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:49.725286 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b88afc-30ce-4fa0-89f7-139f49906fdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:49.725593 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b88afc-30ce-4fa0-89f7-139f49906fdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:49.726984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b88afc-30ce-4fa0-89f7-139f49906fdb None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:49.744520 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 549/1103] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:49.745757 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd0133-57e0-4a9a-bd5e-81f946f6b216 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:49.845175 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd0133-57e0-4a9a-bd5e-81f946f6b216 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:49.881175 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970f1a9-5168-4bc9-bb06-6d8f281ece4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:49.881966 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970f1a9-5168-4bc9-bb06-6d8f281ece4c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:49.882251 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970f1a9-5168-4bc9-bb06-6d8f281ece4c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:49.882527 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970f1a9-5168-4bc9-bb06-6d8f281ece4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:49.889333 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970f1a9-5168-4bc9-bb06-6d8f281ece4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:49.890786 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970f1a9-5168-4bc9-bb06-6d8f281ece4c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:49.905099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-4970f1a9-5168-4bc9-bb06-6d8f281ece4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:49.905099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:26:49.905099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:26:49.905099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:26:49.905099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:26:49.905099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:26:49.905099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:26:49.905099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:26:49.905099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:26:49.905099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:26:49.905099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:26:49.905099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:26:49.905099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:26:49.905099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:26:49.905099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:26:49.906287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:26:49.906287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:26:49.906287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:26:49.906287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:26:49.906287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:26:49.906287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:26:49.906287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:26:49.906287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:26:49.906287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:26:49.906287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:26:49.906287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:26:49.906287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:26:49.906287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:26:49.906287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:26:49.906287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:26:49.906287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:26:49.906287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:26:49.907245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:26:49.907245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:26:49.907245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:26:49.907245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:26:49.907245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:26:49.907245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:26:49.907245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:26:49.907245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:26:49.907245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 555/1104] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:26:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:26:49.912833 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b88afc-30ce-4fa0-89f7-139f49906fdb None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:50.014348 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b88afc-30ce-4fa0-89f7-139f49906fdb None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:50.048781 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4a2b5-8a64-4993-83f4-ca378c2d3842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:50.050213 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4a2b5-8a64-4993-83f4-ca378c2d3842 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:50.050495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4a2b5-8a64-4993-83f4-ca378c2d3842 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:50.050770 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4a2b5-8a64-4993-83f4-ca378c2d3842 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:50.051561 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4a2b5-8a64-4993-83f4-ca378c2d3842 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:50.051900 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4a2b5-8a64-4993-83f4-ca378c2d3842 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:50.053329 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4a2b5-8a64-4993-83f4-ca378c2d3842 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:50.065815 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 550/1105] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:26:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:50.072021 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970f1a9-5168-4bc9-bb06-6d8f281ece4c None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:50.103361 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:50.173592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970f1a9-5168-4bc9-bb06-6d8f281ece4c None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:50.208023 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724186c-bbf0-49c5-a02e-c8c70e0ae730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:50.208683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724186c-bbf0-49c5-a02e-c8c70e0ae730 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:50.208967 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724186c-bbf0-49c5-a02e-c8c70e0ae730 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:50.209242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724186c-bbf0-49c5-a02e-c8c70e0ae730 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:50.210072 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4724186c-bbf0-49c5-a02e-c8c70e0ae730 None admin] RBAC: Adding query filter params (role.id=9fd88609ddd942d5a9b4bc684e5e49b2, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=84cba474fc7c470ea119de697b9d7ef9) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:50.210380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4724186c-bbf0-49c5-a02e-c8c70e0ae730 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:50.211836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4724186c-bbf0-49c5-a02e-c8c70e0ae730 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:50.218087 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 556/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:26:50 2022] GET /identity/v3/role_assignments?user.id=84cba474fc7c470ea119de697b9d7ef9&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=9fd88609ddd942d5a9b4bc684e5e49b2 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:50.531890 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cef15e-6028-438b-9f15-3590e28ba3fa None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:50.532182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cef15e-6028-438b-9f15-3590e28ba3fa None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:50.532457 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cef15e-6028-438b-9f15-3590e28ba3fa None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:50.534644 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 551/1107] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:50.540068 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ccfe4b-bd0b-4f8e-9762-ba373d31b07e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:50.540357 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ccfe4b-bd0b-4f8e-9762-ba373d31b07e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:50.540642 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ccfe4b-bd0b-4f8e-9762-ba373d31b07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:50.559118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d2ccfe4b-bd0b-4f8e-9762-ba373d31b07e None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:50.560394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d2ccfe4b-bd0b-4f8e-9762-ba373d31b07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:50.640293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 557/1108] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:50.666194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cef15e-6028-438b-9f15-3590e28ba3fa None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:50.684164 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:50.780562 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cef15e-6028-438b-9f15-3590e28ba3fa None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:50.821988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f1c32-003f-4a37-bec9-0976682b9ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:50.822713 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f1c32-003f-4a37-bec9-0976682b9ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:50.822990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f1c32-003f-4a37-bec9-0976682b9ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:50.823261 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f1c32-003f-4a37-bec9-0976682b9ad1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:50.824510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f1c32-003f-4a37-bec9-0976682b9ad1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:50.825954 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f1c32-003f-4a37-bec9-0976682b9ad1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:50.827629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 552/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:26:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:26:50.834741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ccfe4b-bd0b-4f8e-9762-ba373d31b07e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:50.873991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:50.962368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ccfe4b-bd0b-4f8e-9762-ba373d31b07e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:50.999215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0b3c7-ad28-4b65-b75c-1438b138ac34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:50.999935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0b3c7-ad28-4b65-b75c-1438b138ac34 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:51.000214 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0b3c7-ad28-4b65-b75c-1438b138ac34 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:51.000488 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0b3c7-ad28-4b65-b75c-1438b138ac34 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:51.001225 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0b3c7-ad28-4b65-b75c-1438b138ac34 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:51.003588 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0b3c7-ad28-4b65-b75c-1438b138ac34 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:51.122121 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 558/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 18 08:26:50 2022] POST /identity/v3/users => generated 286 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:26:53.213942 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d5b59-e174-4cd2-abb9-be986dcca625 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:53.214480 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d5b59-e174-4cd2-abb9-be986dcca625 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:53.214480 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d5b59-e174-4cd2-abb9-be986dcca625 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:53.215897 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 553/1111] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:53.222188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb8c92-e8b1-4c3f-94b1-9cc09a39d05b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:53.222725 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb8c92-e8b1-4c3f-94b1-9cc09a39d05b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:53.223047 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb8c92-e8b1-4c3f-94b1-9cc09a39d05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:53.241509 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-98cb8c92-e8b1-4c3f-94b1-9cc09a39d05b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:53.242714 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-98cb8c92-e8b1-4c3f-94b1-9cc09a39d05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:53.316178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 559/1112] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:26:53.340366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d5b59-e174-4cd2-abb9-be986dcca625 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:53.428155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:53.447430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d5b59-e174-4cd2-abb9-be986dcca625 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:53.485797 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59592aed-9330-4e72-9534-c95692c6f95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:53.487182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59592aed-9330-4e72-9534-c95692c6f95a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:53.487379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59592aed-9330-4e72-9534-c95692c6f95a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:53.487586 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59592aed-9330-4e72-9534-c95692c6f95a None admin] PATH_INFO: `/v3/roles/d2ba35e60f974010a5e7dd5c4bebecf8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:53.489161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59592aed-9330-4e72-9534-c95692c6f95a None admin] RBAC: Authorizing `identity:get_role(role_id=d2ba35e60f974010a5e7dd5c4bebecf8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:53.490542 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59592aed-9330-4e72-9534-c95692c6f95a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:53.492003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 554/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:26:53 2022] GET /identity/v3/roles/d2ba35e60f974010a5e7dd5c4bebecf8 => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:53.499940 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb8c92-e8b1-4c3f-94b1-9cc09a39d05b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:53.514899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:53.609730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb8c92-e8b1-4c3f-94b1-9cc09a39d05b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:53.649197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4309d-843a-4c7b-9181-40a58e4a1149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:53.650083 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4309d-843a-4c7b-9181-40a58e4a1149 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:53.650364 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4309d-843a-4c7b-9181-40a58e4a1149 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:53.650644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4309d-843a-4c7b-9181-40a58e4a1149 None admin] PATH_INFO: `/v3/users/c494b95f3620430ebf6e2967ca1ec861` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:53.665079 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4309d-843a-4c7b-9181-40a58e4a1149 None admin] RBAC: Authorizing `identity:get_user(user_id=c494b95f3620430ebf6e2967ca1ec861)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:53.666560 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4309d-843a-4c7b-9181-40a58e4a1149 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:53.668384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 560/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:26:53 2022] GET /identity/v3/users/c494b95f3620430ebf6e2967ca1ec861 => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:53.674322 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59592aed-9330-4e72-9534-c95692c6f95a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:53.779692 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59592aed-9330-4e72-9534-c95692c6f95a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:53.816503 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34162af-ca42-43d1-bb61-5b1b8703d397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:53.817167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34162af-ca42-43d1-bb61-5b1b8703d397 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:53.817446 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34162af-ca42-43d1-bb61-5b1b8703d397 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:53.817719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34162af-ca42-43d1-bb61-5b1b8703d397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:53.841601 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:53.898549 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34162af-ca42-43d1-bb61-5b1b8703d397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:53.900042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34162af-ca42-43d1-bb61-5b1b8703d397 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:54.004209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 555/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:26:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 08:26:54.010308 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4309d-843a-4c7b-9181-40a58e4a1149 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:54.082903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:54.114582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4309d-843a-4c7b-9181-40a58e4a1149 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:54.148299 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31293a02-450a-42ae-b106-b745d8f6449b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:54.149145 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31293a02-450a-42ae-b106-b745d8f6449b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:54.149460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31293a02-450a-42ae-b106-b745d8f6449b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:54.149727 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31293a02-450a-42ae-b106-b745d8f6449b None admin] PATH_INFO: `/v3/projects/2a3be74e0193419cb22409cb92e171e0` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:54.150909 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31293a02-450a-42ae-b106-b745d8f6449b None admin] RBAC: Authorizing `identity:get_project(project_id=2a3be74e0193419cb22409cb92e171e0)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:54.152365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31293a02-450a-42ae-b106-b745d8f6449b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:54.154042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 561/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:26:54 2022] GET /identity/v3/projects/2a3be74e0193419cb22409cb92e171e0 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:54.161001 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34162af-ca42-43d1-bb61-5b1b8703d397 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:54.211840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:54.261341 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34162af-ca42-43d1-bb61-5b1b8703d397 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:54.294241 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e5f02-ebcd-4b24-831d-ae1dc4a2085b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:54.294889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e5f02-ebcd-4b24-831d-ae1dc4a2085b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:54.295173 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e5f02-ebcd-4b24-831d-ae1dc4a2085b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:54.295446 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e5f02-ebcd-4b24-831d-ae1dc4a2085b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:54.296305 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e5f02-ebcd-4b24-831d-ae1dc4a2085b None admin] RBAC: Adding query filter params (role.id=d2ba35e60f974010a5e7dd5c4bebecf8, scope.project.id=2a3be74e0193419cb22409cb92e171e0, user.id=c494b95f3620430ebf6e2967ca1ec861) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:26:54.296612 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e5f02-ebcd-4b24-831d-ae1dc4a2085b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:54.298044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e5f02-ebcd-4b24-831d-ae1dc4a2085b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:54.304272 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 556/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:26:54 2022] GET /identity/v3/role_assignments?user.id=c494b95f3620430ebf6e2967ca1ec861&scope.project.id=2a3be74e0193419cb22409cb92e171e0&role.id=d2ba35e60f974010a5e7dd5c4bebecf8 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:56.305822 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f94fb2-2f2e-43b6-ac7a-36c919b714b6 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:56.306356 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f94fb2-2f2e-43b6-ac7a-36c919b714b6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:56.306356 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f94fb2-2f2e-43b6-ac7a-36c919b714b6 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:56.307874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 562/1118] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:56.313031 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ad6e0-dc50-46a6-aaad-4e1c2483e425 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:56.313426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ad6e0-dc50-46a6-aaad-4e1c2483e425 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:56.313582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ad6e0-dc50-46a6-aaad-4e1c2483e425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:56.331373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-e30ad6e0-dc50-46a6-aaad-4e1c2483e425 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:56.332701 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e30ad6e0-dc50-46a6-aaad-4e1c2483e425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:56.398900 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 557/1119] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:56.422090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f94fb2-2f2e-43b6-ac7a-36c919b714b6 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:56.523106 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f94fb2-2f2e-43b6-ac7a-36c919b714b6 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:56.558482 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4565fa-3f81-41f3-bb98-916a3a0b4a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:56.559286 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4565fa-3f81-41f3-bb98-916a3a0b4a8a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:56.559565 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4565fa-3f81-41f3-bb98-916a3a0b4a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:56.559866 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4565fa-3f81-41f3-bb98-916a3a0b4a8a None admin] PATH_INFO: `/v3/roles/d2ba35e60f974010a5e7dd5c4bebecf8` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:56.561336 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4565fa-3f81-41f3-bb98-916a3a0b4a8a None admin] RBAC: Authorizing `identity:get_role(role_id=d2ba35e60f974010a5e7dd5c4bebecf8)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:56.562700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4565fa-3f81-41f3-bb98-916a3a0b4a8a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:56.564194 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 563/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:26:56 2022] GET /identity/v3/roles/d2ba35e60f974010a5e7dd5c4bebecf8 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:56.570182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ad6e0-dc50-46a6-aaad-4e1c2483e425 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:56.615490 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:56.669069 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ad6e0-dc50-46a6-aaad-4e1c2483e425 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:56.703430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6236f99-b06f-4f3d-98f2-5a860cc0f089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:56.704338 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6236f99-b06f-4f3d-98f2-5a860cc0f089 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:56.704646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6236f99-b06f-4f3d-98f2-5a860cc0f089 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:56.705465 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6236f99-b06f-4f3d-98f2-5a860cc0f089 None admin] PATH_INFO: `/v3/users/c494b95f3620430ebf6e2967ca1ec861` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:56.706686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6236f99-b06f-4f3d-98f2-5a860cc0f089 None admin] RBAC: Authorizing `identity:get_user(user_id=c494b95f3620430ebf6e2967ca1ec861)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:56.708150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6236f99-b06f-4f3d-98f2-5a860cc0f089 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:56.709804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 558/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:26:56 2022] GET /identity/v3/users/c494b95f3620430ebf6e2967ca1ec861 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:56.715650 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4565fa-3f81-41f3-bb98-916a3a0b4a8a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:56.725163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:56.817649 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4565fa-3f81-41f3-bb98-916a3a0b4a8a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:56.850921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8e17b-da46-4178-bf32-b0102b92472e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:56.851816 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8e17b-da46-4178-bf32-b0102b92472e None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:56.852099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8e17b-da46-4178-bf32-b0102b92472e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:56.852385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8e17b-da46-4178-bf32-b0102b92472e None admin] PATH_INFO: `/v3/projects/2a3be74e0193419cb22409cb92e171e0` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:56.853542 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8e17b-da46-4178-bf32-b0102b92472e None admin] RBAC: Authorizing `identity:get_project(project_id=2a3be74e0193419cb22409cb92e171e0)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:56.854973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8e17b-da46-4178-bf32-b0102b92472e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:56.856699 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 564/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:26:56 2022] GET /identity/v3/projects/2a3be74e0193419cb22409cb92e171e0 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:56.863892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6236f99-b06f-4f3d-98f2-5a860cc0f089 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:56.966135 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6236f99-b06f-4f3d-98f2-5a860cc0f089 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:56.979717 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:57.000656 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb21cc6-d7bc-4989-b248-6dedb416980d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:57.002065 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb21cc6-d7bc-4989-b248-6dedb416980d None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:57.002910 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb21cc6-d7bc-4989-b248-6dedb416980d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:57.003185 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb21cc6-d7bc-4989-b248-6dedb416980d None admin] PATH_INFO: `/v3/projects/2a3be74e0193419cb22409cb92e171e0/users/c494b95f3620430ebf6e2967ca1ec861/roles/d2ba35e60f974010a5e7dd5c4bebecf8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:57.005350 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb21cc6-d7bc-4989-b248-6dedb416980d None admin] RBAC: Authorizing `identity:create_grant(project_id=2a3be74e0193419cb22409cb92e171e0, user_id=c494b95f3620430ebf6e2967ca1ec861, role_id=d2ba35e60f974010a5e7dd5c4bebecf8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:57.016668 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb21cc6-d7bc-4989-b248-6dedb416980d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:57.098714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 559/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:26:56 2022] PUT /identity/v3/projects/2a3be74e0193419cb22409cb92e171e0/users/c494b95f3620430ebf6e2967ca1ec861/roles/d2ba35e60f974010a5e7dd5c4bebecf8 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:26:59.082784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5565a-9142-4a24-87ac-ffef79ea48f1 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:59.083341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5565a-9142-4a24-87ac-ffef79ea48f1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:59.083341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5565a-9142-4a24-87ac-ffef79ea48f1 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:59.084730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 565/1124] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:26:59.090004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431e1e5-7f90-4d82-8bf5-df88f1207813 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:59.090403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431e1e5-7f90-4d82-8bf5-df88f1207813 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:59.090616 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431e1e5-7f90-4d82-8bf5-df88f1207813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:59.108127 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-f431e1e5-7f90-4d82-8bf5-df88f1207813 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:26:59.109297 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f431e1e5-7f90-4d82-8bf5-df88f1207813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:26:59.174467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 560/1125] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:26:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:26:59.197156 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5565a-9142-4a24-87ac-ffef79ea48f1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:59.277791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:59.304062 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5565a-9142-4a24-87ac-ffef79ea48f1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:59.338304 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172994a0-b642-4f51-8407-78dc37c61676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:59.339168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172994a0-b642-4f51-8407-78dc37c61676 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:59.339455 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172994a0-b642-4f51-8407-78dc37c61676 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:59.339759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172994a0-b642-4f51-8407-78dc37c61676 None admin] PATH_INFO: `/v3/roles/d2ba35e60f974010a5e7dd5c4bebecf8` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:59.341327 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172994a0-b642-4f51-8407-78dc37c61676 None admin] RBAC: Authorizing `identity:get_role(role_id=d2ba35e60f974010a5e7dd5c4bebecf8)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:59.342656 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172994a0-b642-4f51-8407-78dc37c61676 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:59.344066 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 566/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:26:59 2022] GET /identity/v3/roles/d2ba35e60f974010a5e7dd5c4bebecf8 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:59.350609 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431e1e5-7f90-4d82-8bf5-df88f1207813 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:59.449591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431e1e5-7f90-4d82-8bf5-df88f1207813 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:59.451109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:59.483494 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71db0a0-2b94-4b9b-9f76-3422e521dbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:59.484391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71db0a0-2b94-4b9b-9f76-3422e521dbb0 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:59.484674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71db0a0-2b94-4b9b-9f76-3422e521dbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:59.484949 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71db0a0-2b94-4b9b-9f76-3422e521dbb0 None admin] PATH_INFO: `/v3/users/c494b95f3620430ebf6e2967ca1ec861` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:59.487231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71db0a0-2b94-4b9b-9f76-3422e521dbb0 None admin] RBAC: Authorizing `identity:get_user(user_id=c494b95f3620430ebf6e2967ca1ec861)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:59.488752 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71db0a0-2b94-4b9b-9f76-3422e521dbb0 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:59.490427 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 561/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:26:59 2022] GET /identity/v3/users/c494b95f3620430ebf6e2967ca1ec861 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:59.496376 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172994a0-b642-4f51-8407-78dc37c61676 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:59.602730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172994a0-b642-4f51-8407-78dc37c61676 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:59.637976 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f24436-8958-405b-b17d-e47bec45df44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:59.638629 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f24436-8958-405b-b17d-e47bec45df44 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:59.638917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f24436-8958-405b-b17d-e47bec45df44 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:59.639197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f24436-8958-405b-b17d-e47bec45df44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:59.651429 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:26:59.707360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f24436-8958-405b-b17d-e47bec45df44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:59.708856 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f24436-8958-405b-b17d-e47bec45df44 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:59.807660 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 567/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:26:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 08:26:59.813639 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71db0a0-2b94-4b9b-9f76-3422e521dbb0 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:26:59.915542 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71db0a0-2b94-4b9b-9f76-3422e521dbb0 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:26:59.949850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155a917-2d35-4d8d-bc6c-3221c2983d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:26:59.950699 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155a917-2d35-4d8d-bc6c-3221c2983d38 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:26:59.950979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155a917-2d35-4d8d-bc6c-3221c2983d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:26:59.951252 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155a917-2d35-4d8d-bc6c-3221c2983d38 None admin] PATH_INFO: `/v3/projects/2a3be74e0193419cb22409cb92e171e0` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:26:59.952452 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155a917-2d35-4d8d-bc6c-3221c2983d38 None admin] RBAC: Authorizing `identity:get_project(project_id=2a3be74e0193419cb22409cb92e171e0)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:26:59.953870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155a917-2d35-4d8d-bc6c-3221c2983d38 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:26:59.955612 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 562/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:26:59 2022] GET /identity/v3/projects/2a3be74e0193419cb22409cb92e171e0 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:26:59.961860 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f24436-8958-405b-b17d-e47bec45df44 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:00.005916 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:00.064389 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f24436-8958-405b-b17d-e47bec45df44 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:00.099375 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f39c71-f7f7-4da2-a69f-ac35ca46768a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:00.100065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f39c71-f7f7-4da2-a69f-ac35ca46768a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:00.100335 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f39c71-f7f7-4da2-a69f-ac35ca46768a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:00.100606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f39c71-f7f7-4da2-a69f-ac35ca46768a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:00.101431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f39c71-f7f7-4da2-a69f-ac35ca46768a None admin] RBAC: Adding query filter params (role.id=d2ba35e60f974010a5e7dd5c4bebecf8, scope.project.id=2a3be74e0193419cb22409cb92e171e0, user.id=c494b95f3620430ebf6e2967ca1ec861) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:27:00.101740 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f39c71-f7f7-4da2-a69f-ac35ca46768a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:00.103163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f39c71-f7f7-4da2-a69f-ac35ca46768a None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:00.109157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 568/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:26:59 2022] GET /identity/v3/role_assignments?user.id=c494b95f3620430ebf6e2967ca1ec861&scope.project.id=2a3be74e0193419cb22409cb92e171e0&role.id=d2ba35e60f974010a5e7dd5c4bebecf8 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:02.212717 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8ea7c-3e24-4d7b-b912-1c6a1f164518 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:02.213259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8ea7c-3e24-4d7b-b912-1c6a1f164518 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:02.213398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8ea7c-3e24-4d7b-b912-1c6a1f164518 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:02.214778 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 563/1131] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:27:02.220604 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b441d-a44d-4811-b1dd-1547dfea6a72 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:02.221039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b441d-a44d-4811-b1dd-1547dfea6a72 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:02.221152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b441d-a44d-4811-b1dd-1547dfea6a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:02.238383 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a45b441d-a44d-4811-b1dd-1547dfea6a72 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:27:02.239579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a45b441d-a44d-4811-b1dd-1547dfea6a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:27:02.307115 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 569/1132] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:27:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:27:02.330262 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8ea7c-3e24-4d7b-b912-1c6a1f164518 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:02.372714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:02.434656 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8ea7c-3e24-4d7b-b912-1c6a1f164518 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:02.468221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d09a36-7440-4b90-9425-124e56ddf5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:02.468928 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d09a36-7440-4b90-9425-124e56ddf5e1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:02.469212 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d09a36-7440-4b90-9425-124e56ddf5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:02.469491 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d09a36-7440-4b90-9425-124e56ddf5e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:02.471309 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d09a36-7440-4b90-9425-124e56ddf5e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:02.472776 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d09a36-7440-4b90-9425-124e56ddf5e1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:02.474403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 564/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:27:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:02.480707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b441d-a44d-4811-b1dd-1547dfea6a72 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:02.514207 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:02.583496 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b441d-a44d-4811-b1dd-1547dfea6a72 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:02.620770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94287629-1ca4-450e-8844-1313737b4f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:02.621439 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94287629-1ca4-450e-8844-1313737b4f04 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:02.621724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94287629-1ca4-450e-8844-1313737b4f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:02.622001 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94287629-1ca4-450e-8844-1313737b4f04 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:02.622720 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94287629-1ca4-450e-8844-1313737b4f04 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:02.624276 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94287629-1ca4-450e-8844-1313737b4f04 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:02.711518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 570/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 18 08:27:02 2022] POST /identity/v3/projects => generated 311 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:27:04.767888 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b68f8-f620-4e46-93f9-c9e9c2e89c50 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:04.768511 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b68f8-f620-4e46-93f9-c9e9c2e89c50 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:04.768511 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b68f8-f620-4e46-93f9-c9e9c2e89c50 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:04.769809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 565/1135] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:27:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:27:04.775203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed3a95-b4b8-4c97-abf8-7225060e6337 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:04.775635 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed3a95-b4b8-4c97-abf8-7225060e6337 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:04.775768 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed3a95-b4b8-4c97-abf8-7225060e6337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:04.792726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-deed3a95-b4b8-4c97-abf8-7225060e6337 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:27:04.793895 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-deed3a95-b4b8-4c97-abf8-7225060e6337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:27:04.859278 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 571/1136] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:27:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:27:04.883800 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b68f8-f620-4e46-93f9-c9e9c2e89c50 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:04.984100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b68f8-f620-4e46-93f9-c9e9c2e89c50 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:04.992272 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:05.017846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a8406-5c13-40bf-8c73-acaef4652324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:05.018691 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a8406-5c13-40bf-8c73-acaef4652324 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:05.018918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a8406-5c13-40bf-8c73-acaef4652324 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:05.019189 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a8406-5c13-40bf-8c73-acaef4652324 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:05.020347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a8406-5c13-40bf-8c73-acaef4652324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:05.021775 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a8406-5c13-40bf-8c73-acaef4652324 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:05.023443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 566/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:27:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:05.030536 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed3a95-b4b8-4c97-abf8-7225060e6337 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:05.049940 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:05.130284 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed3a95-b4b8-4c97-abf8-7225060e6337 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:05.164002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f0161-99b9-4b73-b6e5-c68952b1fa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:05.164668 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f0161-99b9-4b73-b6e5-c68952b1fa72 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:05.164952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f0161-99b9-4b73-b6e5-c68952b1fa72 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:05.165229 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f0161-99b9-4b73-b6e5-c68952b1fa72 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:05.165976 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f0161-99b9-4b73-b6e5-c68952b1fa72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:05.167412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f0161-99b9-4b73-b6e5-c68952b1fa72 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:05.253392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 572/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 18 08:27:05 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:27:07.356787 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a2c39-e80f-4a68-a2c0-f29958fb58e8 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:07.357321 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a2c39-e80f-4a68-a2c0-f29958fb58e8 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:07.357321 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a2c39-e80f-4a68-a2c0-f29958fb58e8 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:07.358698 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 567/1139] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:27:07.364153 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77caaf53-6e7c-413a-92bb-3820490b8e57 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:07.364592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77caaf53-6e7c-413a-92bb-3820490b8e57 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:07.364716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77caaf53-6e7c-413a-92bb-3820490b8e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:07.381781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-77caaf53-6e7c-413a-92bb-3820490b8e57 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:27:07.382949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-77caaf53-6e7c-413a-92bb-3820490b8e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:27:07.448737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 573/1140] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:27:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:27:07.472292 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a2c39-e80f-4a68-a2c0-f29958fb58e8 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:07.572584 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a2c39-e80f-4a68-a2c0-f29958fb58e8 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:07.606262 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed1233-c167-4e57-bcc1-8ba3ea5485e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:07.607114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed1233-c167-4e57-bcc1-8ba3ea5485e3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:07.607342 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed1233-c167-4e57-bcc1-8ba3ea5485e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:07.607612 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed1233-c167-4e57-bcc1-8ba3ea5485e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:07.618273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed1233-c167-4e57-bcc1-8ba3ea5485e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:27:07.626501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-0bed1233-c167-4e57-bcc1-8ba3ea5485e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:27:07.626501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:27:07.626501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:27:07.626501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:27:07.626501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:27:07.626501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:27:07.626501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:27:07.626501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:27:07.626501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:27:07.626501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:27:07.626501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:27:07.626501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:27:07.626501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:27:07.626501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:27:07.626501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:27:07.626501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:27:07.627633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:27:07.627633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:27:07.627633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:27:07.627633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:27:07.627633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:27:07.627633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:27:07.627633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:27:07.627633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:27:07.627633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:27:07.627633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:27:07.627633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:27:07.627633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:27:07.627633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:27:07.627633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:27:07.627633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:27:07.627633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:27:07.628579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:27:07.628579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:27:07.628579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:27:07.628579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:27:07.628579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:27:07.628579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:27:07.628579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:27:07.628579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:27:07.628579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:27:07.628579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:27:07.628579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:27:07.628579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:27:07.628579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 568/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:27:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:27:07.633876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77caaf53-6e7c-413a-92bb-3820490b8e57 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:07.637039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:07.732601 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77caaf53-6e7c-413a-92bb-3820490b8e57 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:07.767557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b7927-c5a0-46c9-ac71-5daa17f02851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:07.768429 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b7927-c5a0-46c9-ac71-5daa17f02851 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:07.768720 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b7927-c5a0-46c9-ac71-5daa17f02851 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:07.769003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b7927-c5a0-46c9-ac71-5daa17f02851 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:07.769760 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b7927-c5a0-46c9-ac71-5daa17f02851 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:27:07.770073 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b7927-c5a0-46c9-ac71-5daa17f02851 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:07.771456 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b7927-c5a0-46c9-ac71-5daa17f02851 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:07.779640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 574/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:27:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:07.785678 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed1233-c167-4e57-bcc1-8ba3ea5485e3 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:07.820370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:07.886985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed1233-c167-4e57-bcc1-8ba3ea5485e3 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:07.921054 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59547a36-67b9-462b-91ef-9faec4a5d703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:07.921914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59547a36-67b9-462b-91ef-9faec4a5d703 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:07.922195 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59547a36-67b9-462b-91ef-9faec4a5d703 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:07.922467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59547a36-67b9-462b-91ef-9faec4a5d703 None admin] PATH_INFO: `/v3/users/3c89b3248bf445aebeeae0ee02678cd0` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:07.936299 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59547a36-67b9-462b-91ef-9faec4a5d703 None admin] RBAC: Authorizing `identity:get_user(user_id=3c89b3248bf445aebeeae0ee02678cd0)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:07.937727 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59547a36-67b9-462b-91ef-9faec4a5d703 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:07.939779 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 569/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:27:07 2022] GET /identity/v3/users/3c89b3248bf445aebeeae0ee02678cd0 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:07.945691 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b7927-c5a0-46c9-ac71-5daa17f02851 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:08.010892 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:08.046451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b7927-c5a0-46c9-ac71-5daa17f02851 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:08.079891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eac7cb-8b3d-4894-b208-7e0d7853ead9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:08.080538 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eac7cb-8b3d-4894-b208-7e0d7853ead9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:08.080825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eac7cb-8b3d-4894-b208-7e0d7853ead9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:08.081103 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eac7cb-8b3d-4894-b208-7e0d7853ead9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:08.148596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eac7cb-8b3d-4894-b208-7e0d7853ead9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:08.149969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eac7cb-8b3d-4894-b208-7e0d7853ead9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:08.214263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:08.251375 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 575/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:27:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 08:27:08.257581 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59547a36-67b9-462b-91ef-9faec4a5d703 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:08.350355 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:08.358870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59547a36-67b9-462b-91ef-9faec4a5d703 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:08.396148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af843754-a323-43a9-ade7-581102113cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:08.396991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af843754-a323-43a9-ade7-581102113cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:08.397263 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af843754-a323-43a9-ade7-581102113cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:08.397534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af843754-a323-43a9-ade7-581102113cb3 None admin] PATH_INFO: `/v3/projects/77512776dab745089d8a2b1535ce292a` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:08.398762 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af843754-a323-43a9-ade7-581102113cb3 None admin] RBAC: Authorizing `identity:get_project(project_id=77512776dab745089d8a2b1535ce292a)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:08.400153 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af843754-a323-43a9-ade7-581102113cb3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:08.401850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 570/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:27:08 2022] GET /identity/v3/projects/77512776dab745089d8a2b1535ce292a => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:08.408791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eac7cb-8b3d-4894-b208-7e0d7853ead9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:08.507185 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eac7cb-8b3d-4894-b208-7e0d7853ead9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:08.541752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1740776-589b-4ed9-bd7c-ca52b7ce9f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:08.542406 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1740776-589b-4ed9-bd7c-ca52b7ce9f16 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:08.542699 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1740776-589b-4ed9-bd7c-ca52b7ce9f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:08.542975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1740776-589b-4ed9-bd7c-ca52b7ce9f16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:08.543844 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1740776-589b-4ed9-bd7c-ca52b7ce9f16 None admin] RBAC: Adding query filter params (role.id=467d49b8a0d3435aaba58464cc8ef85c, scope.project.id=77512776dab745089d8a2b1535ce292a, user.id=3c89b3248bf445aebeeae0ee02678cd0) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:27:08.544157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1740776-589b-4ed9-bd7c-ca52b7ce9f16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:08.545595 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1740776-589b-4ed9-bd7c-ca52b7ce9f16 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:08.551121 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 576/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:27:08 2022] GET /identity/v3/role_assignments?user.id=3c89b3248bf445aebeeae0ee02678cd0&scope.project.id=77512776dab745089d8a2b1535ce292a&role.id=467d49b8a0d3435aaba58464cc8ef85c => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:10.554083 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afc0fe-f1c0-4fad-8ccf-a7160d116c34 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:10.554640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afc0fe-f1c0-4fad-8ccf-a7160d116c34 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:10.554640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afc0fe-f1c0-4fad-8ccf-a7160d116c34 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:10.556051 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 571/1147] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:27:10.561163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c48a30-2ddb-4ef8-8c33-c7ba39818d2b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:10.561584 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c48a30-2ddb-4ef8-8c33-c7ba39818d2b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:10.561726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c48a30-2ddb-4ef8-8c33-c7ba39818d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:10.579818 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-80c48a30-2ddb-4ef8-8c33-c7ba39818d2b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:27:10.581013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-80c48a30-2ddb-4ef8-8c33-c7ba39818d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:27:10.604510 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:10.648527 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 577/1148] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:27:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:27:10.670545 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91afc0fe-f1c0-4fad-8ccf-a7160d116c34 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:10.772374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91afc0fe-f1c0-4fad-8ccf-a7160d116c34 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:10.806702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17dc4af-0730-44b5-9d89-a7d1272921d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:10.807406 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dc4af-0730-44b5-9d89-a7d1272921d8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:10.807704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dc4af-0730-44b5-9d89-a7d1272921d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:10.807986 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dc4af-0730-44b5-9d89-a7d1272921d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:10.818201 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17dc4af-0730-44b5-9d89-a7d1272921d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:27:10.825741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-e17dc4af-0730-44b5-9d89-a7d1272921d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:27:10.825741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:27:10.825741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:27:10.825741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:27:10.825741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:27:10.825741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:27:10.825741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:27:10.825741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:27:10.825741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:27:10.825741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:27:10.825741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:27:10.825741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:27:10.825741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:27:10.825741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:27:10.825741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:27:10.825741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:27:10.826824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:27:10.826824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:27:10.826824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:27:10.826824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:27:10.826824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:27:10.826824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:27:10.826824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:27:10.826824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:27:10.826824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:27:10.826824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:27:10.826824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:27:10.826824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:27:10.826824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:27:10.826824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:27:10.826824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:27:10.826824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:27:10.827735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:27:10.827735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:27:10.827735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:27:10.827735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:27:10.827735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:27:10.827735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:27:10.827735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:27:10.827735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:27:10.827735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:27:10.827735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:27:10.827735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:27:10.827735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:27:10.827735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 572/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:27:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:27:10.833721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c48a30-2ddb-4ef8-8c33-c7ba39818d2b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:10.934275 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c48a30-2ddb-4ef8-8c33-c7ba39818d2b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:10.969638 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db78ccf-8e7b-4b25-ae3f-492be4681767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:10.970389 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db78ccf-8e7b-4b25-ae3f-492be4681767 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:10.970672 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db78ccf-8e7b-4b25-ae3f-492be4681767 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:10.970963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db78ccf-8e7b-4b25-ae3f-492be4681767 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:10.971726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db78ccf-8e7b-4b25-ae3f-492be4681767 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:27:10.972038 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db78ccf-8e7b-4b25-ae3f-492be4681767 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:10.973421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db78ccf-8e7b-4b25-ae3f-492be4681767 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:10.981667 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 578/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:27:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:10.988126 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17dc4af-0730-44b5-9d89-a7d1272921d8 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:11.011924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:11.093328 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17dc4af-0730-44b5-9d89-a7d1272921d8 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:11.127908 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3c2f0-5bf9-41ef-b042-f05affa6ff14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:11.128764 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3c2f0-5bf9-41ef-b042-f05affa6ff14 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:11.129042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3c2f0-5bf9-41ef-b042-f05affa6ff14 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:11.129313 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3c2f0-5bf9-41ef-b042-f05affa6ff14 None admin] PATH_INFO: `/v3/users/3c89b3248bf445aebeeae0ee02678cd0` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:11.130468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3c2f0-5bf9-41ef-b042-f05affa6ff14 None admin] RBAC: Authorizing `identity:get_user(user_id=3c89b3248bf445aebeeae0ee02678cd0)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:11.131948 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3c2f0-5bf9-41ef-b042-f05affa6ff14 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:11.133574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 573/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:27:10 2022] GET /identity/v3/users/3c89b3248bf445aebeeae0ee02678cd0 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:11.139460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db78ccf-8e7b-4b25-ae3f-492be4681767 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:11.166626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:11.247247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db78ccf-8e7b-4b25-ae3f-492be4681767 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:11.284997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7411b-6f1c-425b-91d7-2cd21b4781b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:11.286008 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7411b-6f1c-425b-91d7-2cd21b4781b2 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:11.286431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7411b-6f1c-425b-91d7-2cd21b4781b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:11.296941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7411b-6f1c-425b-91d7-2cd21b4781b2 None admin] PATH_INFO: `/v3/projects/77512776dab745089d8a2b1535ce292a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:11.296941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7411b-6f1c-425b-91d7-2cd21b4781b2 None admin] RBAC: Authorizing `identity:get_project(project_id=77512776dab745089d8a2b1535ce292a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:11.296941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7411b-6f1c-425b-91d7-2cd21b4781b2 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:11.296941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 579/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:27:11 2022] GET /identity/v3/projects/77512776dab745089d8a2b1535ce292a => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:11.298603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3c2f0-5bf9-41ef-b042-f05affa6ff14 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:11.387941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:11.399941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3c2f0-5bf9-41ef-b042-f05affa6ff14 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:11.434464 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a947c5-c0e2-4be0-a377-a02f4aee80c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:11.435353 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a947c5-c0e2-4be0-a377-a02f4aee80c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:11.435638 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a947c5-c0e2-4be0-a377-a02f4aee80c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:11.435955 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a947c5-c0e2-4be0-a377-a02f4aee80c3 None admin] PATH_INFO: `/v3/projects/77512776dab745089d8a2b1535ce292a/users/3c89b3248bf445aebeeae0ee02678cd0/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:11.437923 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a947c5-c0e2-4be0-a377-a02f4aee80c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=77512776dab745089d8a2b1535ce292a, user_id=3c89b3248bf445aebeeae0ee02678cd0, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:11.439967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a947c5-c0e2-4be0-a377-a02f4aee80c3 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:11.515127 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 574/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:27:11 2022] PUT /identity/v3/projects/77512776dab745089d8a2b1535ce292a/users/3c89b3248bf445aebeeae0ee02678cd0/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:27:13.559224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062bd73-fa3a-40af-a9f4-7b39184dc6c3 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:13.559909 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062bd73-fa3a-40af-a9f4-7b39184dc6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:13.559909 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062bd73-fa3a-40af-a9f4-7b39184dc6c3 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:13.561185 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 580/1154] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:27:13.567069 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b365cd1-399f-4641-a387-902eb66b9ce4 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:13.567534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b365cd1-399f-4641-a387-902eb66b9ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:13.567633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b365cd1-399f-4641-a387-902eb66b9ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:13.584744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2b365cd1-399f-4641-a387-902eb66b9ce4 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:27:13.585941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2b365cd1-399f-4641-a387-902eb66b9ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:27:13.654379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 575/1155] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:27:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:27:13.678710 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062bd73-fa3a-40af-a9f4-7b39184dc6c3 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:13.769865 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:13.780096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062bd73-fa3a-40af-a9f4-7b39184dc6c3 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:13.819046 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776ab6e-486d-4b06-b942-9cd4bb0a8f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:13.819797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776ab6e-486d-4b06-b942-9cd4bb0a8f1a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:13.820082 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776ab6e-486d-4b06-b942-9cd4bb0a8f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:13.820356 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776ab6e-486d-4b06-b942-9cd4bb0a8f1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:13.831615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776ab6e-486d-4b06-b942-9cd4bb0a8f1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:27:13.839294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-0776ab6e-486d-4b06-b942-9cd4bb0a8f1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:27:13.839294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:27:13.839294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:27:13.839294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:27:13.839294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:27:13.839294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:27:13.839294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:27:13.839294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:27:13.839294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:27:13.839294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:27:13.839294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:27:13.839294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:27:13.839294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:27:13.839294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:27:13.839294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:27:13.839294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:27:13.840514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:27:13.840514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:27:13.840514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:27:13.840514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:27:13.840514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:27:13.840514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:27:13.840514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:27:13.840514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:27:13.840514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:27:13.840514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:27:13.840514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:27:13.840514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:27:13.840514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:27:13.840514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:27:13.840514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:27:13.840514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:27:13.841643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:27:13.841643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:27:13.841643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:27:13.841643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:27:13.841643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:27:13.841643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:27:13.841643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:27:13.841643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:27:13.841643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:27:13.841643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:27:13.841643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:27:13.841643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:27:13.841643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 581/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:27:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:27:13.848036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b365cd1-399f-4641-a387-902eb66b9ce4 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:13.927661 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:13.949789 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b365cd1-399f-4641-a387-902eb66b9ce4 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:13.985796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afba6e-987d-44f8-9960-2fa7fccc630e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:13.986642 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afba6e-987d-44f8-9960-2fa7fccc630e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:13.986925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afba6e-987d-44f8-9960-2fa7fccc630e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:13.987197 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afba6e-987d-44f8-9960-2fa7fccc630e None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:13.987985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afba6e-987d-44f8-9960-2fa7fccc630e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:27:13.988288 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afba6e-987d-44f8-9960-2fa7fccc630e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:13.989760 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afba6e-987d-44f8-9960-2fa7fccc630e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:13.999000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 576/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:27:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:14.005397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776ab6e-486d-4b06-b942-9cd4bb0a8f1a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:14.106893 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776ab6e-486d-4b06-b942-9cd4bb0a8f1a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:14.140544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b14e0c-13a8-44b8-a618-5102ff294bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:14.141979 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b14e0c-13a8-44b8-a618-5102ff294bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:14.142263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b14e0c-13a8-44b8-a618-5102ff294bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:14.142632 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b14e0c-13a8-44b8-a618-5102ff294bd5 None admin] PATH_INFO: `/v3/users/3c89b3248bf445aebeeae0ee02678cd0` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:14.143828 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b14e0c-13a8-44b8-a618-5102ff294bd5 None admin] RBAC: Authorizing `identity:get_user(user_id=3c89b3248bf445aebeeae0ee02678cd0)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:14.145298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b14e0c-13a8-44b8-a618-5102ff294bd5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:14.146986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 582/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:27:14 2022] GET /identity/v3/users/3c89b3248bf445aebeeae0ee02678cd0 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:14.153076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afba6e-987d-44f8-9960-2fa7fccc630e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:14.254330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afba6e-987d-44f8-9960-2fa7fccc630e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:14.287781 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2768d1b-c0de-43be-87d1-cfe2e791404f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:14.288430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2768d1b-c0de-43be-87d1-cfe2e791404f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:14.288707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2768d1b-c0de-43be-87d1-cfe2e791404f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:14.288986 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2768d1b-c0de-43be-87d1-cfe2e791404f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:14.300895 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:14.356778 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2768d1b-c0de-43be-87d1-cfe2e791404f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:14.358269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2768d1b-c0de-43be-87d1-cfe2e791404f None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:14.461246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 577/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:27:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 08:27:14.467366 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b14e0c-13a8-44b8-a618-5102ff294bd5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:14.488594 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:14.574673 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b14e0c-13a8-44b8-a618-5102ff294bd5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:14.610214 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2847c5-bea4-4400-aec6-d70da11b9b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:14.611108 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2847c5-bea4-4400-aec6-d70da11b9b30 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:14.611388 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2847c5-bea4-4400-aec6-d70da11b9b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:14.611658 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2847c5-bea4-4400-aec6-d70da11b9b30 None admin] PATH_INFO: `/v3/projects/77512776dab745089d8a2b1535ce292a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:14.612864 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2847c5-bea4-4400-aec6-d70da11b9b30 None admin] RBAC: Authorizing `identity:get_project(project_id=77512776dab745089d8a2b1535ce292a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:14.614293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2847c5-bea4-4400-aec6-d70da11b9b30 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:14.616020 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 583/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:27:14 2022] GET /identity/v3/projects/77512776dab745089d8a2b1535ce292a => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:14.622350 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2768d1b-c0de-43be-87d1-cfe2e791404f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:14.664308 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:14.721621 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2768d1b-c0de-43be-87d1-cfe2e791404f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:14.755793 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0e29b-ee6b-4cea-872a-339d10b566d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:14.756446 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0e29b-ee6b-4cea-872a-339d10b566d1 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:14.756731 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0e29b-ee6b-4cea-872a-339d10b566d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:14.757013 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0e29b-ee6b-4cea-872a-339d10b566d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:14.757841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0e29b-ee6b-4cea-872a-339d10b566d1 None admin] RBAC: Adding query filter params (role.id=467d49b8a0d3435aaba58464cc8ef85c, scope.project.id=77512776dab745089d8a2b1535ce292a, user.id=3c89b3248bf445aebeeae0ee02678cd0) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:27:14.758153 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0e29b-ee6b-4cea-872a-339d10b566d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:14.759586 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0e29b-ee6b-4cea-872a-339d10b566d1 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:14.765387 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 578/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:27:14 2022] GET /identity/v3/role_assignments?user.id=3c89b3248bf445aebeeae0ee02678cd0&scope.project.id=77512776dab745089d8a2b1535ce292a&role.id=467d49b8a0d3435aaba58464cc8ef85c => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:16.791065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7efcdc-1e06-47a5-b37f-c192899ca7cb None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:16.791630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7efcdc-1e06-47a5-b37f-c192899ca7cb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:16.791630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7efcdc-1e06-47a5-b37f-c192899ca7cb None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:16.793010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 584/1162] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:27:16.799127 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408e4c4-324a-49bb-bbe2-dd2fecfc840b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:16.799590 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408e4c4-324a-49bb-bbe2-dd2fecfc840b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:16.799714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408e4c4-324a-49bb-bbe2-dd2fecfc840b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:16.816608 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2408e4c4-324a-49bb-bbe2-dd2fecfc840b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:27:16.817796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2408e4c4-324a-49bb-bbe2-dd2fecfc840b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:27:16.887896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 579/1163] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:27:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:27:16.928088 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7efcdc-1e06-47a5-b37f-c192899ca7cb None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:17.058917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:17.075126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7efcdc-1e06-47a5-b37f-c192899ca7cb None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:17.110472 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc6d95-8ebb-4ef0-9aad-326c7eb9f0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:17.111131 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc6d95-8ebb-4ef0-9aad-326c7eb9f0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:17.111416 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc6d95-8ebb-4ef0-9aad-326c7eb9f0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:17.111717 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc6d95-8ebb-4ef0-9aad-326c7eb9f0f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:17.179852 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc6d95-8ebb-4ef0-9aad-326c7eb9f0f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:17.181332 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc6d95-8ebb-4ef0-9aad-326c7eb9f0f7 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:17.264818 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:17.280495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 585/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:27:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 08:27:17.287410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408e4c4-324a-49bb-bbe2-dd2fecfc840b None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:17.320755 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:17.386799 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408e4c4-324a-49bb-bbe2-dd2fecfc840b None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:17.420378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89247bc-a0d3-4010-a5a0-ab625d46304d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:17.421194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89247bc-a0d3-4010-a5a0-ab625d46304d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:17.421474 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89247bc-a0d3-4010-a5a0-ab625d46304d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:17.421747 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89247bc-a0d3-4010-a5a0-ab625d46304d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:17.428636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89247bc-a0d3-4010-a5a0-ab625d46304d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:17.430059 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89247bc-a0d3-4010-a5a0-ab625d46304d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:17.443964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-e89247bc-a0d3-4010-a5a0-ab625d46304d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 08:27:17.443964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:27:17.443964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 18 08:27:17.443964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 18 08:27:17.443964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:27:17.443964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:27:17.443964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:27:17.443964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:27:17.443964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 18 08:27:17.443964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:27:17.443964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:27:17.443964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:27:17.443964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:27:17.443964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:27:17.443964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:27:17.443964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:27:17.443964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:27:17.445206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:27:17.445206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:27:17.445206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:27:17.445206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:27:17.445206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:27:17.445206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:27:17.445206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:27:17.445206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:27:17.445206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:27:17.445206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:27:17.445206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:27:17.445206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:27:17.445206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:27:17.445206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:27:17.445206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:27:17.445206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:27:17.446274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:27:17.446274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:27:17.446274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:27:17.446274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:27:17.446274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:27:17.446274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:27:17.446274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:27:17.446274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:27:17.446274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:27:17.446274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:27:17.446274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 18 08:27:17.446274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:27:17.446274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 08:27:17.446274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:27:17.446274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 580/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 18 08:27:17 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:27:17.451266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc6d95-8ebb-4ef0-9aad-326c7eb9f0f7 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:17.560246 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc6d95-8ebb-4ef0-9aad-326c7eb9f0f7 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:17.597573 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6fb5b-4875-43ac-9506-323107fac21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:17.598148 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6fb5b-4875-43ac-9506-323107fac21b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:17.598428 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6fb5b-4875-43ac-9506-323107fac21b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:17.598698 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6fb5b-4875-43ac-9506-323107fac21b None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:17.599443 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6fb5b-4875-43ac-9506-323107fac21b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:27:17.599778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6fb5b-4875-43ac-9506-323107fac21b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:17.601157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6fb5b-4875-43ac-9506-323107fac21b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:17.608903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 586/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Jul 18 08:27:17 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:19.668500 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbccb3-ff7f-4547-812b-2ffde746ab7b None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:19.669042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbccb3-ff7f-4547-812b-2ffde746ab7b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:19.669042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbccb3-ff7f-4547-812b-2ffde746ab7b None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:19.670430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 581/1167] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:27:19.676485 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376a133-eda3-4aa3-8eaf-6f0929db2761 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:19.676957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376a133-eda3-4aa3-8eaf-6f0929db2761 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:19.677086 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376a133-eda3-4aa3-8eaf-6f0929db2761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:19.694790 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3376a133-eda3-4aa3-8eaf-6f0929db2761 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:27:19.696053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3376a133-eda3-4aa3-8eaf-6f0929db2761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:27:19.749285 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:19.771313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 587/1168] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:27:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:27:19.794576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbccb3-ff7f-4547-812b-2ffde746ab7b None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:19.905354 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:19.915414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbccb3-ff7f-4547-812b-2ffde746ab7b None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:19.953046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bd285-f876-4fae-9162-824b3fe3d110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:19.953771 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bd285-f876-4fae-9162-824b3fe3d110 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:19.954055 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bd285-f876-4fae-9162-824b3fe3d110 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:19.954327 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bd285-f876-4fae-9162-824b3fe3d110 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:19.955077 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179bd285-f876-4fae-9162-824b3fe3d110 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:19.956508 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179bd285-f876-4fae-9162-824b3fe3d110 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:20.032783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 582/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Mon Jul 18 08:27:19 2022] POST /identity/v3/domains => generated 267 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:27:22.094153 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105c6a3-3ec1-40bb-b31d-a06984771b0a None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:22.094748 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105c6a3-3ec1-40bb-b31d-a06984771b0a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:22.094748 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105c6a3-3ec1-40bb-b31d-a06984771b0a None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:22.096048 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 588/1170] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:27:22.101814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c9101-187b-4008-a9a6-9bd00dda4e24 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:22.102207 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c9101-187b-4008-a9a6-9bd00dda4e24 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:22.102374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c9101-187b-4008-a9a6-9bd00dda4e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:22.119280 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-153c9101-187b-4008-a9a6-9bd00dda4e24 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:27:22.120492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-153c9101-187b-4008-a9a6-9bd00dda4e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:27:22.185512 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 583/1171] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:27:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:27:22.209572 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105c6a3-3ec1-40bb-b31d-a06984771b0a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:22.310441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105c6a3-3ec1-40bb-b31d-a06984771b0a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:22.345256 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b7954-8a94-48e5-b150-092aa2530fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:22.346039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b7954-8a94-48e5-b150-092aa2530fce None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:22.346325 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b7954-8a94-48e5-b150-092aa2530fce None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:22.346600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b7954-8a94-48e5-b150-092aa2530fce None admin] PATH_INFO: `/v3/domains/eea8b95d80bd488a9d7c9acf2ddc743c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:22.357107 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b7954-8a94-48e5-b150-092aa2530fce None admin] RBAC: Authorizing `identity:get_domain(domain_id=eea8b95d80bd488a9d7c9acf2ddc743c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:22.358562 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b7954-8a94-48e5-b150-092aa2530fce None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:22.360557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 589/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Mon Jul 18 08:27:22 2022] GET /identity/v3/domains/eea8b95d80bd488a9d7c9acf2ddc743c => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:27:22.366585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c9101-187b-4008-a9a6-9bd00dda4e24 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:22.449171 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:22.465052 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c9101-187b-4008-a9a6-9bd00dda4e24 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:22.498736 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db92ace-dcc0-4610-9f00-02c880a8779e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:22.499391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db92ace-dcc0-4610-9f00-02c880a8779e None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:22.499698 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db92ace-dcc0-4610-9f00-02c880a8779e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:22.500471 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db92ace-dcc0-4610-9f00-02c880a8779e None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:22.501172 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db92ace-dcc0-4610-9f00-02c880a8779e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:22.502600 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db92ace-dcc0-4610-9f00-02c880a8779e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:22.590867 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 584/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 18 08:27:22 2022] POST /identity/v3/projects => generated 361 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:27:24.636835 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245f6a1-8652-4076-9b61-a841d097e3ae None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:24.637404 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245f6a1-8652-4076-9b61-a841d097e3ae None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:24.637404 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245f6a1-8652-4076-9b61-a841d097e3ae None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:24.638774 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 590/1174] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:27:24.645458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83420682-1676-45c6-958f-0740c4d0946c None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:24.646004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83420682-1676-45c6-958f-0740c4d0946c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:24.646004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83420682-1676-45c6-958f-0740c4d0946c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:24.664302 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-83420682-1676-45c6-958f-0740c4d0946c None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:27:24.665493 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-83420682-1676-45c6-958f-0740c4d0946c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:27:24.731883 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 585/1175] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:27:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:27:24.756662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245f6a1-8652-4076-9b61-a841d097e3ae None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:24.802007 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:24.865569 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245f6a1-8652-4076-9b61-a841d097e3ae None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:24.899182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23508b8b-1a54-4d52-ba3b-2d013db2cfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:24.900005 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23508b8b-1a54-4d52-ba3b-2d013db2cfba None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:24.900288 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23508b8b-1a54-4d52-ba3b-2d013db2cfba None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:24.900559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23508b8b-1a54-4d52-ba3b-2d013db2cfba None admin] PATH_INFO: `/v3/domains/eea8b95d80bd488a9d7c9acf2ddc743c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:24.901728 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23508b8b-1a54-4d52-ba3b-2d013db2cfba None admin] RBAC: Authorizing `identity:get_domain(domain_id=eea8b95d80bd488a9d7c9acf2ddc743c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:24.903152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23508b8b-1a54-4d52-ba3b-2d013db2cfba None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:24.904778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 591/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Mon Jul 18 08:27:24 2022] GET /identity/v3/domains/eea8b95d80bd488a9d7c9acf2ddc743c => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:24.911694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83420682-1676-45c6-958f-0740c4d0946c None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:24.976229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:25.011115 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83420682-1676-45c6-958f-0740c4d0946c None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:25.045851 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327dc8a6-5063-405c-bff6-f7cfd84e16cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:25.046530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327dc8a6-5063-405c-bff6-f7cfd84e16cc None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:25.047282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327dc8a6-5063-405c-bff6-f7cfd84e16cc None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:25.047282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327dc8a6-5063-405c-bff6-f7cfd84e16cc None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:25.047864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327dc8a6-5063-405c-bff6-f7cfd84e16cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:25.049291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327dc8a6-5063-405c-bff6-f7cfd84e16cc None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:25.109803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 586/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 18 08:27:24 2022] POST /identity/v3/users => generated 311 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:27:27.166061 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac86925-3bde-4cd1-83fe-e0173508122b None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:27.166566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac86925-3bde-4cd1-83fe-e0173508122b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:27.166566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac86925-3bde-4cd1-83fe-e0173508122b None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:27.167911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 592/1178] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:27:27.173792 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14668c-4a13-4923-ba0e-db29c652f375 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:27.174182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14668c-4a13-4923-ba0e-db29c652f375 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:27.174357 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14668c-4a13-4923-ba0e-db29c652f375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:27.191312 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-9d14668c-4a13-4923-ba0e-db29c652f375 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:27:27.192533 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9d14668c-4a13-4923-ba0e-db29c652f375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:27:27.258197 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 587/1179] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:27:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:27:27.281940 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac86925-3bde-4cd1-83fe-e0173508122b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:27.380488 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac86925-3bde-4cd1-83fe-e0173508122b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:27.389649 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:27.413428 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94946b7-c1eb-4b32-8e64-20f4909d3d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:27.414210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94946b7-c1eb-4b32-8e64-20f4909d3d14 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:27.414492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94946b7-c1eb-4b32-8e64-20f4909d3d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:27.414768 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94946b7-c1eb-4b32-8e64-20f4909d3d14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:27.425259 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94946b7-c1eb-4b32-8e64-20f4909d3d14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:27:27.432755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-e94946b7-c1eb-4b32-8e64-20f4909d3d14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:27:27.432755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:27:27.432755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:27:27.432755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:27:27.432755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:27:27.432755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:27:27.432755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:27:27.432755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:27:27.432755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:27:27.432755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:27:27.432755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:27:27.432755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:27:27.432755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:27:27.432755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:27:27.432755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:27:27.432755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:27:27.433873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:27:27.433873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:27:27.433873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:27:27.433873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:27:27.433873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:27:27.433873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:27:27.433873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:27:27.433873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:27:27.433873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:27:27.433873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:27:27.433873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:27:27.433873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:27:27.433873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:27:27.433873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:27:27.433873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:27:27.433873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:27:27.435100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:27:27.435100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:27:27.435100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:27:27.435100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:27:27.435100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:27:27.435100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:27:27.435100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:27:27.435100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:27:27.435100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:27:27.435100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:27:27.435100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:27:27.435100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:27:27.435100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 593/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:27:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:27:27.440652 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14668c-4a13-4923-ba0e-db29c652f375 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:27.491933 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:27.538445 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14668c-4a13-4923-ba0e-db29c652f375 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:27.572890 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c67335-a2ef-47da-a09a-d1e7ead555a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:27.573556 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c67335-a2ef-47da-a09a-d1e7ead555a8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:27.573836 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c67335-a2ef-47da-a09a-d1e7ead555a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:27.574110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c67335-a2ef-47da-a09a-d1e7ead555a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:27.574854 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c67335-a2ef-47da-a09a-d1e7ead555a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:27:27.575159 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c67335-a2ef-47da-a09a-d1e7ead555a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:27.576554 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c67335-a2ef-47da-a09a-d1e7ead555a8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:27.585662 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 588/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:27:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:27.591570 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94946b7-c1eb-4b32-8e64-20f4909d3d14 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:27.693693 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94946b7-c1eb-4b32-8e64-20f4909d3d14 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:27.728784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0e50e-8e3f-4e34-875d-72ee505c0056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:27.729655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0e50e-8e3f-4e34-875d-72ee505c0056 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:27.729948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0e50e-8e3f-4e34-875d-72ee505c0056 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:27.730226 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0e50e-8e3f-4e34-875d-72ee505c0056 None admin] PATH_INFO: `/v3/users/702403a4150342c1b71f14ae1ad0f653` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:27.744704 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0e50e-8e3f-4e34-875d-72ee505c0056 None admin] RBAC: Authorizing `identity:get_user(user_id=702403a4150342c1b71f14ae1ad0f653)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:27.746169 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0e50e-8e3f-4e34-875d-72ee505c0056 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:27.747934 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 594/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:27:27 2022] GET /identity/v3/users/702403a4150342c1b71f14ae1ad0f653 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:27.754512 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c67335-a2ef-47da-a09a-d1e7ead555a8 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:27.857910 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c67335-a2ef-47da-a09a-d1e7ead555a8 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:27.873315 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:27.893887 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a432b9-e9df-4eed-a5c7-fdda4d8abc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:27.894590 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a432b9-e9df-4eed-a5c7-fdda4d8abc71 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:27.894915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a432b9-e9df-4eed-a5c7-fdda4d8abc71 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:27.895189 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a432b9-e9df-4eed-a5c7-fdda4d8abc71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:27.962085 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a432b9-e9df-4eed-a5c7-fdda4d8abc71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:27.963445 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a432b9-e9df-4eed-a5c7-fdda4d8abc71 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:28.063654 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 589/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:27:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 08:27:28.069561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0e50e-8e3f-4e34-875d-72ee505c0056 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:28.098960 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:28.172737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0e50e-8e3f-4e34-875d-72ee505c0056 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:28.207249 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa572e-f997-4fd8-bbf7-dc43136cb0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:28.208136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa572e-f997-4fd8-bbf7-dc43136cb0b5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:28.208419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa572e-f997-4fd8-bbf7-dc43136cb0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:28.208692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa572e-f997-4fd8-bbf7-dc43136cb0b5 None admin] PATH_INFO: `/v3/projects/af27e3b8458c4eb4844ab5a09beb1b68` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:28.209867 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa572e-f997-4fd8-bbf7-dc43136cb0b5 None admin] RBAC: Authorizing `identity:get_project(project_id=af27e3b8458c4eb4844ab5a09beb1b68)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:28.211296 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa572e-f997-4fd8-bbf7-dc43136cb0b5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:28.213111 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 595/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:27:28 2022] GET /identity/v3/projects/af27e3b8458c4eb4844ab5a09beb1b68 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:28.219220 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a432b9-e9df-4eed-a5c7-fdda4d8abc71 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:28.234112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:28.319699 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a432b9-e9df-4eed-a5c7-fdda4d8abc71 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:28.354509 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7aefbd-1ea6-42f9-8e1c-49708ff52d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:28.355157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7aefbd-1ea6-42f9-8e1c-49708ff52d96 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:28.355436 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7aefbd-1ea6-42f9-8e1c-49708ff52d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:28.355733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7aefbd-1ea6-42f9-8e1c-49708ff52d96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:28.356569 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7aefbd-1ea6-42f9-8e1c-49708ff52d96 None admin] RBAC: Adding query filter params (role.id=467d49b8a0d3435aaba58464cc8ef85c, scope.project.id=af27e3b8458c4eb4844ab5a09beb1b68, user.id=702403a4150342c1b71f14ae1ad0f653) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:27:28.356877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7aefbd-1ea6-42f9-8e1c-49708ff52d96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:28.358312 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7aefbd-1ea6-42f9-8e1c-49708ff52d96 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:28.364933 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 590/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:27:28 2022] GET /identity/v3/role_assignments?user.id=702403a4150342c1b71f14ae1ad0f653&scope.project.id=af27e3b8458c4eb4844ab5a09beb1b68&role.id=467d49b8a0d3435aaba58464cc8ef85c => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:30.370482 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa75a4-d0e6-4617-acc4-d993078c3fcb None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:30.371046 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa75a4-d0e6-4617-acc4-d993078c3fcb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:30.371046 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa75a4-d0e6-4617-acc4-d993078c3fcb None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:30.372441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 596/1186] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:27:30.378636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d6364-73fd-442d-94fa-1771cc1895e2 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:30.379053 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d6364-73fd-442d-94fa-1771cc1895e2 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:30.379191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d6364-73fd-442d-94fa-1771cc1895e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:30.395873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-aa5d6364-73fd-442d-94fa-1771cc1895e2 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:27:30.397063 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-aa5d6364-73fd-442d-94fa-1771cc1895e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:27:30.462520 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 591/1187] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:27:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:27:30.486114 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa75a4-d0e6-4617-acc4-d993078c3fcb None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:30.585903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:30.593385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa75a4-d0e6-4617-acc4-d993078c3fcb None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:30.628395 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d063e-4980-4b17-b3c7-4a2f0432a9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:30.629085 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d063e-4980-4b17-b3c7-4a2f0432a9dd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:30.629367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d063e-4980-4b17-b3c7-4a2f0432a9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:30.629641 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d063e-4980-4b17-b3c7-4a2f0432a9dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:30.639866 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d063e-4980-4b17-b3c7-4a2f0432a9dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:27:30.648484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-6b7d063e-4980-4b17-b3c7-4a2f0432a9dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:27:30.648484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:27:30.648484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:27:30.648484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:27:30.648484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:27:30.648484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:27:30.648484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:27:30.648484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:27:30.648484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:27:30.648484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:27:30.648484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:27:30.648484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:27:30.648484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:27:30.648484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:27:30.648484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:27:30.648484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:27:30.649615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:27:30.649615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:27:30.649615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:27:30.649615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:27:30.649615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:27:30.649615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:27:30.649615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:27:30.649615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:27:30.649615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:27:30.649615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:27:30.649615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:27:30.649615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:27:30.649615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:27:30.649615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:27:30.649615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:27:30.649615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:27:30.650665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:27:30.650665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:27:30.650665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:27:30.650665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:27:30.650665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:27:30.650665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:27:30.650665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:27:30.650665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:27:30.650665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:27:30.650665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:27:30.650665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:27:30.650665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:27:30.650665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 597/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:27:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:27:30.655904 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d6364-73fd-442d-94fa-1771cc1895e2 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:30.666089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:30.756740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d6364-73fd-442d-94fa-1771cc1895e2 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:30.790282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5d7f3-4e04-45eb-a938-85a8aab22929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:30.790952 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5d7f3-4e04-45eb-a938-85a8aab22929 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:30.791235 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5d7f3-4e04-45eb-a938-85a8aab22929 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:30.791555 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5d7f3-4e04-45eb-a938-85a8aab22929 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:30.792312 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5d7f3-4e04-45eb-a938-85a8aab22929 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:27:30.792604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5d7f3-4e04-45eb-a938-85a8aab22929 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:30.793967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5d7f3-4e04-45eb-a938-85a8aab22929 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:30.802968 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 592/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:27:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:30.809019 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d063e-4980-4b17-b3c7-4a2f0432a9dd None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:30.912412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d063e-4980-4b17-b3c7-4a2f0432a9dd None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:30.946380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb26d9c-f755-482f-a0b3-fed3b3f7dc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:30.947244 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb26d9c-f755-482f-a0b3-fed3b3f7dc06 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:30.947530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb26d9c-f755-482f-a0b3-fed3b3f7dc06 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:30.947839 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb26d9c-f755-482f-a0b3-fed3b3f7dc06 None admin] PATH_INFO: `/v3/users/702403a4150342c1b71f14ae1ad0f653` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:30.949052 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb26d9c-f755-482f-a0b3-fed3b3f7dc06 None admin] RBAC: Authorizing `identity:get_user(user_id=702403a4150342c1b71f14ae1ad0f653)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:30.950496 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb26d9c-f755-482f-a0b3-fed3b3f7dc06 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:30.952171 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 598/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:27:30 2022] GET /identity/v3/users/702403a4150342c1b71f14ae1ad0f653 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:30.958579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5d7f3-4e04-45eb-a938-85a8aab22929 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:31.026830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:31.057291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5d7f3-4e04-45eb-a938-85a8aab22929 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:31.094695 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a1b8b-630a-4633-8d34-71344b8594bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:31.095571 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a1b8b-630a-4633-8d34-71344b8594bc None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:31.095915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a1b8b-630a-4633-8d34-71344b8594bc None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:31.096159 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a1b8b-630a-4633-8d34-71344b8594bc None admin] PATH_INFO: `/v3/projects/af27e3b8458c4eb4844ab5a09beb1b68` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:31.097384 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a1b8b-630a-4633-8d34-71344b8594bc None admin] RBAC: Authorizing `identity:get_project(project_id=af27e3b8458c4eb4844ab5a09beb1b68)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:31.098871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a1b8b-630a-4633-8d34-71344b8594bc None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:31.100730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 593/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:27:30 2022] GET /identity/v3/projects/af27e3b8458c4eb4844ab5a09beb1b68 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:31.107324 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb26d9c-f755-482f-a0b3-fed3b3f7dc06 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:31.135113 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:31.213011 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb26d9c-f755-482f-a0b3-fed3b3f7dc06 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:31.247293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff44f71-2125-4e7c-bc56-47d84b3eaceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:31.248205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff44f71-2125-4e7c-bc56-47d84b3eaceb None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:31.248484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff44f71-2125-4e7c-bc56-47d84b3eaceb None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:31.248758 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff44f71-2125-4e7c-bc56-47d84b3eaceb None admin] PATH_INFO: `/v3/projects/af27e3b8458c4eb4844ab5a09beb1b68/users/702403a4150342c1b71f14ae1ad0f653/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:31.250854 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff44f71-2125-4e7c-bc56-47d84b3eaceb None admin] RBAC: Authorizing `identity:create_grant(project_id=af27e3b8458c4eb4844ab5a09beb1b68, user_id=702403a4150342c1b71f14ae1ad0f653, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:31.252309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff44f71-2125-4e7c-bc56-47d84b3eaceb None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:31.332569 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 599/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:27:31 2022] PUT /identity/v3/projects/af27e3b8458c4eb4844ab5a09beb1b68/users/702403a4150342c1b71f14ae1ad0f653/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:27:33.355078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab658e-fe75-4474-866e-356f692ca090 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:33.355612 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab658e-fe75-4474-866e-356f692ca090 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:33.355612 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab658e-fe75-4474-866e-356f692ca090 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:33.357420 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 594/1193] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:27:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:27:33.363730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6a4be-d4a7-457a-b6ad-21f976b6dd0b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:33.364197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6a4be-d4a7-457a-b6ad-21f976b6dd0b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:33.364326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6a4be-d4a7-457a-b6ad-21f976b6dd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:33.382294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-4dc6a4be-d4a7-457a-b6ad-21f976b6dd0b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:27:33.383486 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4dc6a4be-d4a7-457a-b6ad-21f976b6dd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:27:33.450747 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 600/1194] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:27:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:27:33.474774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab658e-fe75-4474-866e-356f692ca090 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:33.579051 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab658e-fe75-4474-866e-356f692ca090 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:33.614112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8812d1f-0938-4c6b-80a0-1bc4b232d71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:33.614819 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8812d1f-0938-4c6b-80a0-1bc4b232d71f None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:33.615097 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8812d1f-0938-4c6b-80a0-1bc4b232d71f None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:33.615369 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8812d1f-0938-4c6b-80a0-1bc4b232d71f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:33.622239 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:33.626411 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8812d1f-0938-4c6b-80a0-1bc4b232d71f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:27:33.633985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-a8812d1f-0938-4c6b-80a0-1bc4b232d71f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:27:33.633985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:27:33.633985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:27:33.633985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:27:33.633985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:27:33.633985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:27:33.633985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:27:33.633985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:27:33.633985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:27:33.633985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:27:33.633985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:27:33.633985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:27:33.633985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:27:33.633985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:27:33.633985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:27:33.633985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:27:33.635932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:27:33.635932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:27:33.635932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:27:33.635932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:27:33.635932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:27:33.635932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:27:33.635932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:27:33.635932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:27:33.635932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:27:33.635932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:27:33.635932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:27:33.635932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:27:33.635932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:27:33.635932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:27:33.635932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:27:33.635932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:27:33.636877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:27:33.636877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:27:33.636877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:27:33.636877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:27:33.636877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:27:33.636877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:27:33.636877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:27:33.636877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:27:33.636877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:27:33.636877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:27:33.636877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:27:33.636877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:27:33.636877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 595/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:27:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:27:33.641665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6a4be-d4a7-457a-b6ad-21f976b6dd0b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:33.656485 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:33.743794 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6a4be-d4a7-457a-b6ad-21f976b6dd0b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:33.778678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83aaa5-08b7-4d16-beda-0857785e8680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:33.779393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83aaa5-08b7-4d16-beda-0857785e8680 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:33.779709 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83aaa5-08b7-4d16-beda-0857785e8680 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:33.779989 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83aaa5-08b7-4d16-beda-0857785e8680 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:33.781339 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83aaa5-08b7-4d16-beda-0857785e8680 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:27:33.781651 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83aaa5-08b7-4d16-beda-0857785e8680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:33.783029 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83aaa5-08b7-4d16-beda-0857785e8680 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:33.791566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 601/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:27:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:27:33.797765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8812d1f-0938-4c6b-80a0-1bc4b232d71f None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:33.900077 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8812d1f-0938-4c6b-80a0-1bc4b232d71f None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:33.939361 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd7bd6-6cdc-4d7e-a5b0-c61f7856c8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:33.940399 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd7bd6-6cdc-4d7e-a5b0-c61f7856c8fa None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:33.940781 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd7bd6-6cdc-4d7e-a5b0-c61f7856c8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:33.941146 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd7bd6-6cdc-4d7e-a5b0-c61f7856c8fa None admin] PATH_INFO: `/v3/users/702403a4150342c1b71f14ae1ad0f653` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:33.942468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd7bd6-6cdc-4d7e-a5b0-c61f7856c8fa None admin] RBAC: Authorizing `identity:get_user(user_id=702403a4150342c1b71f14ae1ad0f653)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:33.944118 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd7bd6-6cdc-4d7e-a5b0-c61f7856c8fa None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:33.945866 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 596/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Jul 18 08:27:33 2022] GET /identity/v3/users/702403a4150342c1b71f14ae1ad0f653 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:33.951746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83aaa5-08b7-4d16-beda-0857785e8680 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:34.029590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:34.056026 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83aaa5-08b7-4d16-beda-0857785e8680 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:34.091020 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98645f6-43d9-4815-bb46-e874f83bcf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:34.091662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98645f6-43d9-4815-bb46-e874f83bcf4b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:34.091981 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98645f6-43d9-4815-bb46-e874f83bcf4b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:34.092257 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98645f6-43d9-4815-bb46-e874f83bcf4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:34.160358 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98645f6-43d9-4815-bb46-e874f83bcf4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:34.161735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98645f6-43d9-4815-bb46-e874f83bcf4b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:34.235453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:34.260782 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 602/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:27:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 08:27:34.267717 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd7bd6-6cdc-4d7e-a5b0-c61f7856c8fa None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:34.333169 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:34.370434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd7bd6-6cdc-4d7e-a5b0-c61f7856c8fa None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:34.408001 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79237d3-a2e6-431e-b455-dc0b35378ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:34.408886 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79237d3-a2e6-431e-b455-dc0b35378ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:34.409167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79237d3-a2e6-431e-b455-dc0b35378ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:34.409440 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79237d3-a2e6-431e-b455-dc0b35378ae8 None admin] PATH_INFO: `/v3/projects/af27e3b8458c4eb4844ab5a09beb1b68` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:34.410723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79237d3-a2e6-431e-b455-dc0b35378ae8 None admin] RBAC: Authorizing `identity:get_project(project_id=af27e3b8458c4eb4844ab5a09beb1b68)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:34.412239 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79237d3-a2e6-431e-b455-dc0b35378ae8 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:34.414045 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 597/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Jul 18 08:27:34 2022] GET /identity/v3/projects/af27e3b8458c4eb4844ab5a09beb1b68 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:34.420548 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98645f6-43d9-4815-bb46-e874f83bcf4b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:27:34.524343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98645f6-43d9-4815-bb46-e874f83bcf4b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:27:34.559809 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69eb7e-a57c-412f-96ed-c4fc376536cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:27:34.560459 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69eb7e-a57c-412f-96ed-c4fc376536cf None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:27:34.560742 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69eb7e-a57c-412f-96ed-c4fc376536cf None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:27:34.561015 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69eb7e-a57c-412f-96ed-c4fc376536cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:27:34.563092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69eb7e-a57c-412f-96ed-c4fc376536cf None admin] RBAC: Adding query filter params (role.id=467d49b8a0d3435aaba58464cc8ef85c, scope.project.id=af27e3b8458c4eb4844ab5a09beb1b68, user.id=702403a4150342c1b71f14ae1ad0f653) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:27:34.563408 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69eb7e-a57c-412f-96ed-c4fc376536cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:27:34.564903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69eb7e-a57c-412f-96ed-c4fc376536cf None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:27:34.570668 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 603/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:27:34 2022] GET /identity/v3/role_assignments?user.id=702403a4150342c1b71f14ae1ad0f653&scope.project.id=af27e3b8458c4eb4844ab5a09beb1b68&role.id=467d49b8a0d3435aaba58464cc8ef85c => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:27:44.403423 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=33 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:27:44.568051 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=81 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:28:52.074652 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf13708-c1c9-42e8-8366-2a52f3930dcc None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:52.075260 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf13708-c1c9-42e8-8366-2a52f3930dcc None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:52.075260 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf13708-c1c9-42e8-8366-2a52f3930dcc None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:52.076732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 598/1201] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:28:52.082531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbd0c0-db7b-49d1-9d5d-6ea6e0d1506e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:52.083906 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbd0c0-db7b-49d1-9d5d-6ea6e0d1506e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:52.083906 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbd0c0-db7b-49d1-9d5d-6ea6e0d1506e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:52.101095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b0dbd0c0-db7b-49d1-9d5d-6ea6e0d1506e None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:28:52.102219 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b0dbd0c0-db7b-49d1-9d5d-6ea6e0d1506e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:28:52.171901 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 604/1202] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:28:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:28:52.196469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf13708-c1c9-42e8-8366-2a52f3930dcc None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:28:52.301702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf13708-c1c9-42e8-8366-2a52f3930dcc None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:28:52.334284 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0ab57-2da2-4986-b27a-d62bf49f943c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:28:52.335070 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0ab57-2da2-4986-b27a-d62bf49f943c None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:52.335352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0ab57-2da2-4986-b27a-d62bf49f943c None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:52.335623 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0ab57-2da2-4986-b27a-d62bf49f943c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:52.346824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0ab57-2da2-4986-b27a-d62bf49f943c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:28:52.348300 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0ab57-2da2-4986-b27a-d62bf49f943c None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:28:52.370640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-6dc0ab57-2da2-4986-b27a-d62bf49f943c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:28:52.370640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:28:52.370640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:28:52.370640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:28:52.370640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:28:52.370640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:28:52.370640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:28:52.370640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:28:52.370640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:28:52.370640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:28:52.370640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:28:52.370640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:28:52.370640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:28:52.370640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:28:52.370640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:28:52.371864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:28:52.371864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:28:52.371864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:28:52.371864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:28:52.371864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:28:52.371864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:28:52.371864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:28:52.371864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:28:52.371864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:28:52.371864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:28:52.371864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:28:52.371864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:28:52.371864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:28:52.371864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:28:52.371864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:28:52.371864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:28:52.371864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:28:52.373042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:28:52.373042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:28:52.373042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:28:52.373042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:28:52.373042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 599/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:28:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:28:52.373620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbd0c0-db7b-49d1-9d5d-6ea6e0d1506e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:28:52.475450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbd0c0-db7b-49d1-9d5d-6ea6e0d1506e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:28:52.503531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:28:52.509116 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046110b-46d3-4523-996c-0d0773b62498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:28:52.514740 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046110b-46d3-4523-996c-0d0773b62498 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:52.514740 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046110b-46d3-4523-996c-0d0773b62498 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:52.514740 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046110b-46d3-4523-996c-0d0773b62498 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:52.514740 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046110b-46d3-4523-996c-0d0773b62498 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:28:52.514740 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046110b-46d3-4523-996c-0d0773b62498 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:28:52.514740 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046110b-46d3-4523-996c-0d0773b62498 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:28:52.524728 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 605/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:28:52.531308 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0ab57-2da2-4986-b27a-d62bf49f943c None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:28:52.585137 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:28:52.631701 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0ab57-2da2-4986-b27a-d62bf49f943c None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:28:52.665120 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af100f5f-d2df-4a91-b859-245c0cba73d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:28:52.665780 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af100f5f-d2df-4a91-b859-245c0cba73d7 None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:52.666065 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af100f5f-d2df-4a91-b859-245c0cba73d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:52.666340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af100f5f-d2df-4a91-b859-245c0cba73d7 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:52.667046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af100f5f-d2df-4a91-b859-245c0cba73d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:28:52.668507 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af100f5f-d2df-4a91-b859-245c0cba73d7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:28:52.769199 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 600/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Jul 18 08:28:52 2022] POST /identity/v3/users => generated 251 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:28:54.870653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00e3a4-b2b4-4e85-9bc5-ab818ed6e097 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:54.871181 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00e3a4-b2b4-4e85-9bc5-ab818ed6e097 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:54.871181 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00e3a4-b2b4-4e85-9bc5-ab818ed6e097 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:54.872570 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 606/1206] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:28:54.878320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa0f15-8e79-4039-8539-9efb7551b598 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:54.880110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa0f15-8e79-4039-8539-9efb7551b598 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:54.880110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa0f15-8e79-4039-8539-9efb7551b598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:54.896726 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-efaa0f15-8e79-4039-8539-9efb7551b598 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:28:54.897915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-efaa0f15-8e79-4039-8539-9efb7551b598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:28:54.964907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 601/1207] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:28:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:28:54.989318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00e3a4-b2b4-4e85-9bc5-ab818ed6e097 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:28:55.097743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00e3a4-b2b4-4e85-9bc5-ab818ed6e097 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:28:55.133101 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d3798-8c87-4240-acc4-8f80053cda76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:28:55.133888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d3798-8c87-4240-acc4-8f80053cda76 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:55.134170 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d3798-8c87-4240-acc4-8f80053cda76 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:55.134444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d3798-8c87-4240-acc4-8f80053cda76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:55.145076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d3798-8c87-4240-acc4-8f80053cda76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:28:55.152796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-bd4d3798-8c87-4240-acc4-8f80053cda76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:28:55.152796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:28:55.152796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:28:55.152796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:28:55.152796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:28:55.152796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:28:55.152796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:28:55.152796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:28:55.152796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:28:55.152796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:28:55.152796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:28:55.152796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:28:55.152796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:28:55.152796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:28:55.152796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:28:55.152796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:28:55.154392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:28:55.154392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:28:55.154392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:28:55.154392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:28:55.154392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:28:55.154392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:28:55.154392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:28:55.154392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:28:55.154392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:28:55.154392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:28:55.154392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:28:55.154392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:28:55.154392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:28:55.154392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:28:55.154392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:28:55.154392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:28:55.155431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:28:55.155431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:28:55.155431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:28:55.155431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:28:55.155431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:28:55.155431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:28:55.155431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:28:55.155431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:28:55.155431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:28:55.155431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:28:55.155431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:28:55.155431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:28:55.155431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 607/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:28:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:28:55.160639 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa0f15-8e79-4039-8539-9efb7551b598 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:28:55.200018 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:28:55.259126 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa0f15-8e79-4039-8539-9efb7551b598 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:28:55.293431 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2bc25-0dd5-4c1e-9b4d-d26c4c467797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:28:55.294093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2bc25-0dd5-4c1e-9b4d-d26c4c467797 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:55.294374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2bc25-0dd5-4c1e-9b4d-d26c4c467797 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:55.294646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2bc25-0dd5-4c1e-9b4d-d26c4c467797 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:55.295445 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2bc25-0dd5-4c1e-9b4d-d26c4c467797 None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:28:55.295775 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2bc25-0dd5-4c1e-9b4d-d26c4c467797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:28:55.297296 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2bc25-0dd5-4c1e-9b4d-d26c4c467797 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:28:55.305248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 602/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:28:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:28:55.311313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d3798-8c87-4240-acc4-8f80053cda76 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:28:55.369372 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:28:55.421477 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d3798-8c87-4240-acc4-8f80053cda76 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:28:55.455638 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e866450-b93b-4380-be4f-a85a8a4a8c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:28:55.456459 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e866450-b93b-4380-be4f-a85a8a4a8c31 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:55.456745 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e866450-b93b-4380-be4f-a85a8a4a8c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:55.457081 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e866450-b93b-4380-be4f-a85a8a4a8c31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:55.467438 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e866450-b93b-4380-be4f-a85a8a4a8c31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:28:55.468989 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e866450-b93b-4380-be4f-a85a8a4a8c31 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:28:55.486911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-0e866450-b93b-4380-be4f-a85a8a4a8c31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:28:55.486911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:28:55.486911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:28:55.486911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:28:55.486911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:28:55.486911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:28:55.486911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:28:55.486911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:28:55.486911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:28:55.486911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:28:55.486911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:28:55.486911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:28:55.486911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:28:55.486911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:28:55.486911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:28:55.490946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:28:55.490946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:28:55.490946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:28:55.490946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:28:55.490946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:28:55.490946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:28:55.490946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:28:55.490946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:28:55.490946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:28:55.490946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:28:55.490946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:28:55.490946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:28:55.490946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:28:55.490946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:28:55.490946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:28:55.490946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:28:55.490946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:28:55.492136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:28:55.492136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:28:55.492136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:28:55.492136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:28:55.492136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 608/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:28:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:28:55.494409 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2bc25-0dd5-4c1e-9b4d-d26c4c467797 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:28:55.597553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:28:55.600647 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2bc25-0dd5-4c1e-9b4d-d26c4c467797 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:28:55.634808 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd4f2d-9e80-4ce1-9c15-c745aa97822a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:28:55.635399 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd4f2d-9e80-4ce1-9c15-c745aa97822a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:55.635706 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd4f2d-9e80-4ce1-9c15-c745aa97822a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:55.635990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd4f2d-9e80-4ce1-9c15-c745aa97822a None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:55.636702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd4f2d-9e80-4ce1-9c15-c745aa97822a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:28:55.637008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd4f2d-9e80-4ce1-9c15-c745aa97822a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:28:55.638360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd4f2d-9e80-4ce1-9c15-c745aa97822a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:28:55.649737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 603/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:28:55.655369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e866450-b93b-4380-be4f-a85a8a4a8c31 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:28:55.757986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e866450-b93b-4380-be4f-a85a8a4a8c31 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:28:55.759509 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:28:55.792716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6deaa8c-a9d4-4709-89d8-45b6c956bbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:28:55.793506 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6deaa8c-a9d4-4709-89d8-45b6c956bbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:55.793792 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6deaa8c-a9d4-4709-89d8-45b6c956bbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:55.794071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6deaa8c-a9d4-4709-89d8-45b6c956bbd8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:55.801931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6deaa8c-a9d4-4709-89d8-45b6c956bbd8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:28:55.803374 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6deaa8c-a9d4-4709-89d8-45b6c956bbd8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:28:55.818713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-e6deaa8c-a9d4-4709-89d8-45b6c956bbd8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 08:28:55.818713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:28:55.818713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:28:55.818713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:28:55.818713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:28:55.818713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:28:55.818713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:28:55.818713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:28:55.818713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:28:55.818713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:28:55.818713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:28:55.818713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:28:55.818713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:28:55.818713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:28:55.818713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:28:55.818713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:28:55.819957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:28:55.819957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:28:55.819957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:28:55.819957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:28:55.819957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:28:55.819957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:28:55.819957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:28:55.819957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:28:55.819957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:28:55.819957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:28:55.819957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:28:55.819957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:28:55.819957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:28:55.819957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:28:55.819957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:28:55.819957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:28:55.819957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:28:55.819957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:28:55.821292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:28:55.821292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:28:55.821292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:28:55.821292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:28:55.821292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:28:55.821292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:28:55.821292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:28:55.821292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:28:55.821292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 08:28:55.821292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:28:55.821292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 609/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:28:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:28:55.825914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd4f2d-9e80-4ce1-9c15-c745aa97822a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:28:55.928779 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd4f2d-9e80-4ce1-9c15-c745aa97822a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:28:55.963119 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e6cd0-2cce-4111-8bf6-da34b4a27434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:28:55.963823 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e6cd0-2cce-4111-8bf6-da34b4a27434 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:55.964110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e6cd0-2cce-4111-8bf6-da34b4a27434 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:55.964384 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e6cd0-2cce-4111-8bf6-da34b4a27434 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:55.965341 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e6cd0-2cce-4111-8bf6-da34b4a27434 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:28:55.965650 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e6cd0-2cce-4111-8bf6-da34b4a27434 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:28:55.967253 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e6cd0-2cce-4111-8bf6-da34b4a27434 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:28:55.986571 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 604/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Jul 18 08:28:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:28:55.992397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6deaa8c-a9d4-4709-89d8-45b6c956bbd8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:28:56.101392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6deaa8c-a9d4-4709-89d8-45b6c956bbd8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:28:56.145013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb625e-60dd-43de-af0d-6829818bd10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:28:56.145674 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb625e-60dd-43de-af0d-6829818bd10d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:56.145961 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb625e-60dd-43de-af0d-6829818bd10d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:56.146242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb625e-60dd-43de-af0d-6829818bd10d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:56.179891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:28:56.222616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb625e-60dd-43de-af0d-6829818bd10d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:28:56.224782 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb625e-60dd-43de-af0d-6829818bd10d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:28:56.324144 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 610/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:28:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 08:28:56.331035 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e6cd0-2cce-4111-8bf6-da34b4a27434 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:28:56.351182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:28:56.436056 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e6cd0-2cce-4111-8bf6-da34b4a27434 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:28:56.473311 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffeb05-4e71-4703-b6d3-7a7d5597b6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:28:56.474105 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffeb05-4e71-4703-b6d3-7a7d5597b6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:56.474379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffeb05-4e71-4703-b6d3-7a7d5597b6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:56.474648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffeb05-4e71-4703-b6d3-7a7d5597b6f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:56.485623 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ffeb05-4e71-4703-b6d3-7a7d5597b6f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:28:56.487113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ffeb05-4e71-4703-b6d3-7a7d5597b6f4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:28:56.505514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-10ffeb05-4e71-4703-b6d3-7a7d5597b6f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:28:56.505514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:28:56.505514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:28:56.505514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:28:56.505514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:28:56.505514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:28:56.505514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:28:56.505514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:28:56.505514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:28:56.505514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:28:56.505514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:28:56.505514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:28:56.505514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:28:56.505514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:28:56.505514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:28:56.506814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:28:56.506814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:28:56.506814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:28:56.506814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:28:56.506814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:28:56.506814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:28:56.506814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:28:56.506814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:28:56.506814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:28:56.506814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:28:56.506814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:28:56.506814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:28:56.506814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:28:56.506814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:28:56.506814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:28:56.506814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:28:56.506814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:28:56.512914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:28:56.512914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:28:56.512914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:28:56.512914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:28:56.512914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 605/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:28:56.513285 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb625e-60dd-43de-af0d-6829818bd10d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:28:56.578797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:28:56.621541 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb625e-60dd-43de-af0d-6829818bd10d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:28:56.657223 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a79df-c2fc-4714-bd86-690b21b1b5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:28:56.657899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a79df-c2fc-4714-bd86-690b21b1b5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:56.658267 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a79df-c2fc-4714-bd86-690b21b1b5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:56.658550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a79df-c2fc-4714-bd86-690b21b1b5c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:56.659280 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a79df-c2fc-4714-bd86-690b21b1b5c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:28:56.659592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a79df-c2fc-4714-bd86-690b21b1b5c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:28:56.661002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a79df-c2fc-4714-bd86-690b21b1b5c5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:28:56.673174 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 611/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:28:56.679131 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffeb05-4e71-4703-b6d3-7a7d5597b6f4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:28:56.751390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:28:56.781921 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffeb05-4e71-4703-b6d3-7a7d5597b6f4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:28:56.816471 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb283839-5e64-442e-a3d7-8babdcbc5905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:28:56.817280 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb283839-5e64-442e-a3d7-8babdcbc5905 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:56.817551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb283839-5e64-442e-a3d7-8babdcbc5905 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:56.817835 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb283839-5e64-442e-a3d7-8babdcbc5905 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:56.825375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb283839-5e64-442e-a3d7-8babdcbc5905 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:28:56.826834 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb283839-5e64-442e-a3d7-8babdcbc5905 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:28:56.842884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-eb283839-5e64-442e-a3d7-8babdcbc5905 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:28:56.842884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:28:56.842884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:28:56.842884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:28:56.842884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:28:56.842884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:28:56.842884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:28:56.842884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:28:56.842884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:28:56.842884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:28:56.842884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:28:56.842884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:28:56.842884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:28:56.842884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:28:56.842884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:28:56.843965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:28:56.843965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:28:56.843965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:28:56.843965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:28:56.843965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:28:56.843965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:28:56.843965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:28:56.843965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:28:56.843965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:28:56.843965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:28:56.843965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:28:56.843965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:28:56.843965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:28:56.843965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:28:56.843965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:28:56.843965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:28:56.843965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:28:56.845244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:28:56.845244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:28:56.845244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:28:56.845244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:28:56.845244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:28:56.845244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:28:56.845244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:28:56.845244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:28:56.845244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 606/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:28:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:28:56.850114 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a79df-c2fc-4714-bd86-690b21b1b5c5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:28:56.953310 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a79df-c2fc-4714-bd86-690b21b1b5c5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:28:56.975960 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:28:56.988472 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2145041-62c4-4ddb-800d-076ace2d4974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:28:56.989132 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2145041-62c4-4ddb-800d-076ace2d4974 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:56.989417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2145041-62c4-4ddb-800d-076ace2d4974 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:56.989688 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2145041-62c4-4ddb-800d-076ace2d4974 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:56.990517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2145041-62c4-4ddb-800d-076ace2d4974 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:28:56.990824 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2145041-62c4-4ddb-800d-076ace2d4974 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:28:56.992288 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2145041-62c4-4ddb-800d-076ace2d4974 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:28:57.004754 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 612/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:28:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:28:57.010947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb283839-5e64-442e-a3d7-8babdcbc5905 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:28:57.111717 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb283839-5e64-442e-a3d7-8babdcbc5905 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:28:57.142504 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:28:57.148110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae088a2-b145-44fd-bed7-1eec141b6b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:28:57.148774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae088a2-b145-44fd-bed7-1eec141b6b66 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:57.149060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae088a2-b145-44fd-bed7-1eec141b6b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:57.149339 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae088a2-b145-44fd-bed7-1eec141b6b66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:57.150177 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae088a2-b145-44fd-bed7-1eec141b6b66 None admin] RBAC: Adding query filter params (role.id=9fc4c3dcb90945ebb4eb5566e62e7833, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=d606fd7ef269460a8110223932f02d19) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:28:57.150489 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae088a2-b145-44fd-bed7-1eec141b6b66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:28:57.151957 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae088a2-b145-44fd-bed7-1eec141b6b66 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:28:57.157663 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 607/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:28:57 2022] GET /identity/v3/role_assignments?user.id=d606fd7ef269460a8110223932f02d19&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=9fc4c3dcb90945ebb4eb5566e62e7833 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:28:59.203318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af5551-67ab-4c0a-b27f-3cebb25604c0 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:59.203945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af5551-67ab-4c0a-b27f-3cebb25604c0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:59.208138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af5551-67ab-4c0a-b27f-3cebb25604c0 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:59.208138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 613/1220] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:28:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:28:59.211067 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcad4d8-d6ed-4182-8cf4-ee41839c715b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:59.211491 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcad4d8-d6ed-4182-8cf4-ee41839c715b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:59.211617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcad4d8-d6ed-4182-8cf4-ee41839c715b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:59.229435 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2dcad4d8-d6ed-4182-8cf4-ee41839c715b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:28:59.231091 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2dcad4d8-d6ed-4182-8cf4-ee41839c715b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:28:59.302155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 608/1221] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:28:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:28:59.325055 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af5551-67ab-4c0a-b27f-3cebb25604c0 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:28:59.431302 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af5551-67ab-4c0a-b27f-3cebb25604c0 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:28:59.466179 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474befd0-e8c2-4660-99c3-765dfa8cfdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:28:59.466879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474befd0-e8c2-4660-99c3-765dfa8cfdb8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:59.467163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474befd0-e8c2-4660-99c3-765dfa8cfdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:59.467436 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474befd0-e8c2-4660-99c3-765dfa8cfdb8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:59.478163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474befd0-e8c2-4660-99c3-765dfa8cfdb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:28:59.486703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-474befd0-e8c2-4660-99c3-765dfa8cfdb8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:28:59.486703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:28:59.486703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:28:59.486703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:28:59.486703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:28:59.486703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:28:59.486703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:28:59.486703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:28:59.486703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:28:59.486703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:28:59.486703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:28:59.486703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:28:59.486703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:28:59.486703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:28:59.486703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:28:59.486703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:28:59.487899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:28:59.487899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:28:59.487899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:28:59.487899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:28:59.487899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:28:59.487899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:28:59.487899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:28:59.487899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:28:59.487899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:28:59.487899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:28:59.487899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:28:59.487899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:28:59.487899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:28:59.487899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:28:59.487899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:28:59.487899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:28:59.488931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:28:59.488931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:28:59.488931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:28:59.488931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:28:59.488931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:28:59.488931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:28:59.488931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:28:59.488931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:28:59.488931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:28:59.488931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:28:59.488931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:28:59.488931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:28:59.488931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 614/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:28:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:28:59.494684 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcad4d8-d6ed-4182-8cf4-ee41839c715b None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:28:59.600661 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcad4d8-d6ed-4182-8cf4-ee41839c715b None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:28:59.626446 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:28:59.637099 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6ab79-cfc9-4d4f-9e07-fccea7a416b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:28:59.637771 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6ab79-cfc9-4d4f-9e07-fccea7a416b9 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:59.638061 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6ab79-cfc9-4d4f-9e07-fccea7a416b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:59.638334 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6ab79-cfc9-4d4f-9e07-fccea7a416b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:59.639082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6ab79-cfc9-4d4f-9e07-fccea7a416b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:28:59.639400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6ab79-cfc9-4d4f-9e07-fccea7a416b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:28:59.640925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6ab79-cfc9-4d4f-9e07-fccea7a416b9 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:28:59.652005 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 609/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:28:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:28:59.658651 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474befd0-e8c2-4660-99c3-765dfa8cfdb8 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:28:59.704309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:28:59.762042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474befd0-e8c2-4660-99c3-765dfa8cfdb8 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:28:59.795142 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36a74d-8eb6-4f49-86cb-7c965c6d5b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:28:59.796553 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36a74d-8eb6-4f49-86cb-7c965c6d5b61 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:59.796855 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36a74d-8eb6-4f49-86cb-7c965c6d5b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:59.797134 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36a74d-8eb6-4f49-86cb-7c965c6d5b61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:59.808401 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36a74d-8eb6-4f49-86cb-7c965c6d5b61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:28:59.809870 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36a74d-8eb6-4f49-86cb-7c965c6d5b61 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:28:59.827384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-ed36a74d-8eb6-4f49-86cb-7c965c6d5b61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:28:59.827384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:28:59.827384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:28:59.827384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:28:59.827384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:28:59.827384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:28:59.827384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:28:59.827384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:28:59.827384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:28:59.827384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:28:59.827384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:28:59.827384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:28:59.827384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:28:59.827384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:28:59.827384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:28:59.832690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:28:59.832690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:28:59.832690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:28:59.832690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:28:59.832690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:28:59.832690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:28:59.832690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:28:59.832690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:28:59.832690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:28:59.832690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:28:59.832690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:28:59.832690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:28:59.832690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:28:59.832690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:28:59.832690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:28:59.832690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:28:59.832690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:28:59.833836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:28:59.833836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:28:59.833836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:28:59.833836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:28:59.833836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 615/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:28:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:28:59.834855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6ab79-cfc9-4d4f-9e07-fccea7a416b9 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:28:59.939579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6ab79-cfc9-4d4f-9e07-fccea7a416b9 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:28:59.982077 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d1209-4d48-4bd6-b5e5-46cdace6da42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:28:59.982743 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d1209-4d48-4bd6-b5e5-46cdace6da42 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:28:59.983024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d1209-4d48-4bd6-b5e5-46cdace6da42 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:28:59.983995 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d1209-4d48-4bd6-b5e5-46cdace6da42 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:28:59.984779 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d1209-4d48-4bd6-b5e5-46cdace6da42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:28:59.985087 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d1209-4d48-4bd6-b5e5-46cdace6da42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:28:59.986444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d1209-4d48-4bd6-b5e5-46cdace6da42 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:28:59.998365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 610/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:00.004591 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36a74d-8eb6-4f49-86cb-7c965c6d5b61 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:00.100085 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:00.107020 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36a74d-8eb6-4f49-86cb-7c965c6d5b61 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:00.142049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600fc16-1cd5-4455-bb89-f5309749b9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:00.142846 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600fc16-1cd5-4455-bb89-f5309749b9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:00.143129 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600fc16-1cd5-4455-bb89-f5309749b9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:00.143405 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600fc16-1cd5-4455-bb89-f5309749b9f6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:00.150416 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e600fc16-1cd5-4455-bb89-f5309749b9f6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:00.152350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e600fc16-1cd5-4455-bb89-f5309749b9f6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:00.167411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-e600fc16-1cd5-4455-bb89-f5309749b9f6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 08:29:00.167411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:00.167411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:00.167411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:00.167411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:00.167411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:00.167411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:00.167411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:00.167411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:00.167411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:00.167411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:00.167411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:00.167411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:29:00.167411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:29:00.167411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:29:00.167411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:29:00.168578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:00.168578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:00.168578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:29:00.168578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:29:00.168578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:29:00.168578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:29:00.168578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:00.168578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:00.168578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:00.168578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:00.168578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:00.168578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:00.168578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:00.168578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:00.168578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:00.168578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:00.168578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:00.168578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:00.169708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:00.169708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:00.169708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:29:00.169708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:29:00.169708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:29:00.169708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:29:00.169708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:29:00.169708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:29:00.169708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 08:29:00.169708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:29:00.169708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 616/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:29:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:29:00.175156 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d1209-4d48-4bd6-b5e5-46cdace6da42 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:00.223321 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:00.278244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d1209-4d48-4bd6-b5e5-46cdace6da42 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:00.314275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216273c5-8789-484e-81d5-21da027c3d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:00.315038 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216273c5-8789-484e-81d5-21da027c3d62 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:00.315431 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216273c5-8789-484e-81d5-21da027c3d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:00.315835 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216273c5-8789-484e-81d5-21da027c3d62 None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:00.317068 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216273c5-8789-484e-81d5-21da027c3d62 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:00.317501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216273c5-8789-484e-81d5-21da027c3d62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:00.319421 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216273c5-8789-484e-81d5-21da027c3d62 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:00.338481 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 611/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Jul 18 08:29:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:29:00.344608 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600fc16-1cd5-4455-bb89-f5309749b9f6 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:00.447774 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600fc16-1cd5-4455-bb89-f5309749b9f6 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:00.484060 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3ae97-289c-4a17-93a8-904a2c07340b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:00.484919 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3ae97-289c-4a17-93a8-904a2c07340b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:00.485240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3ae97-289c-4a17-93a8-904a2c07340b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:00.485523 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3ae97-289c-4a17-93a8-904a2c07340b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:00.498163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3ae97-289c-4a17-93a8-904a2c07340b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:00.500226 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3ae97-289c-4a17-93a8-904a2c07340b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:00.518908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-c2f3ae97-289c-4a17-93a8-904a2c07340b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:29:00.518908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:00.518908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:00.518908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:00.518908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:00.518908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:00.518908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:00.518908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:00.518908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:00.518908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:00.518908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:00.518908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:00.518908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:29:00.518908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:29:00.518908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:29:00.526347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:29:00.526347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:00.526347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:00.526347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:00.526347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:00.526347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:00.526347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:00.526347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:00.526347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:00.526347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:00.526347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:00.526347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:00.526347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:00.526347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:00.526347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:00.526347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:00.526347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:00.527324 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:29:00.527324 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:29:00.527324 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:29:00.527324 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:29:00.527324 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 617/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:29:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:00.527614 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216273c5-8789-484e-81d5-21da027c3d62 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:00.634108 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216273c5-8789-484e-81d5-21da027c3d62 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:00.636450 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:00.669724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a5058-32a7-4280-9af9-c074c9f66b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:00.670384 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a5058-32a7-4280-9af9-c074c9f66b39 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:00.670729 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a5058-32a7-4280-9af9-c074c9f66b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:00.671280 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a5058-32a7-4280-9af9-c074c9f66b39 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:00.672222 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a5058-32a7-4280-9af9-c074c9f66b39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:00.672537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a5058-32a7-4280-9af9-c074c9f66b39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:00.673940 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a5058-32a7-4280-9af9-c074c9f66b39 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:00.685825 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 612/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:00.692401 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3ae97-289c-4a17-93a8-904a2c07340b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:00.701495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:00.800974 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3ae97-289c-4a17-93a8-904a2c07340b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:00.838964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca58ab-7079-42dc-9413-face2c1ca020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:00.839819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca58ab-7079-42dc-9413-face2c1ca020 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:00.840100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca58ab-7079-42dc-9413-face2c1ca020 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:00.840379 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca58ab-7079-42dc-9413-face2c1ca020 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:00.848269 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ca58ab-7079-42dc-9413-face2c1ca020 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:00.849721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ca58ab-7079-42dc-9413-face2c1ca020 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:00.864557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-55ca58ab-7079-42dc-9413-face2c1ca020 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:29:00.864557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:00.864557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:00.864557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:00.864557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:00.864557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:00.864557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:00.864557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:00.864557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:00.864557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:00.864557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:00.864557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:00.864557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:29:00.864557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:29:00.864557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:29:00.865697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:29:00.865697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:00.865697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:00.865697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:00.865697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:00.865697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:00.865697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:00.865697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:00.865697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:00.865697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:00.865697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:00.865697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:00.865697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:00.865697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:00.865697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:00.865697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:00.865697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:00.866797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:29:00.866797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:29:00.866797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:29:00.866797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:29:00.866797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:29:00.866797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:29:00.866797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:29:00.866797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:29:00.866797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 618/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:29:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:00.872753 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a5058-32a7-4280-9af9-c074c9f66b39 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:00.976574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a5058-32a7-4280-9af9-c074c9f66b39 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:01.014024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad54a6b-4eea-41b5-9e40-b87e91d355c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:01.014874 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad54a6b-4eea-41b5-9e40-b87e91d355c0 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:01.015329 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad54a6b-4eea-41b5-9e40-b87e91d355c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:01.015609 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad54a6b-4eea-41b5-9e40-b87e91d355c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:01.016433 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad54a6b-4eea-41b5-9e40-b87e91d355c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:01.016740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad54a6b-4eea-41b5-9e40-b87e91d355c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:01.018178 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad54a6b-4eea-41b5-9e40-b87e91d355c0 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:01.031805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 613/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:29:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:01.038101 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca58ab-7079-42dc-9413-face2c1ca020 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:01.105892 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:01.142068 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca58ab-7079-42dc-9413-face2c1ca020 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:01.179216 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732263e0-7cd2-49b0-93a2-e39b5447004e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:01.180125 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732263e0-7cd2-49b0-93a2-e39b5447004e None admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:01.180398 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732263e0-7cd2-49b0-93a2-e39b5447004e None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:01.180665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732263e0-7cd2-49b0-93a2-e39b5447004e None admin] PATH_INFO: `/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/d606fd7ef269460a8110223932f02d19/roles/9fc4c3dcb90945ebb4eb5566e62e7833` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:01.195835 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732263e0-7cd2-49b0-93a2-e39b5447004e None admin] RBAC: Authorizing `identity:create_grant(project_id=35c846931fd443fbb83a6fbe0aeaa866, user_id=d606fd7ef269460a8110223932f02d19, role_id=9fc4c3dcb90945ebb4eb5566e62e7833)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:01.197287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732263e0-7cd2-49b0-93a2-e39b5447004e None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:01.293946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 619/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:29:01 2022] PUT /identity/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/d606fd7ef269460a8110223932f02d19/roles/9fc4c3dcb90945ebb4eb5566e62e7833 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:29:03.302543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563dd124-d2fe-451e-b02b-3659394c5e59 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:03.303390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563dd124-d2fe-451e-b02b-3659394c5e59 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:03.303390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563dd124-d2fe-451e-b02b-3659394c5e59 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:03.304527 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 614/1233] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:29:03.310735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fca20-a8a6-4001-bb00-60a03fb5eb2c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:03.312452 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fca20-a8a6-4001-bb00-60a03fb5eb2c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:03.312933 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fca20-a8a6-4001-bb00-60a03fb5eb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:03.339814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-820fca20-a8a6-4001-bb00-60a03fb5eb2c None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:29:03.341822 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-820fca20-a8a6-4001-bb00-60a03fb5eb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:29:03.454742 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 620/1234] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:29:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:29:03.477813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563dd124-d2fe-451e-b02b-3659394c5e59 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:03.504375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:03.587139 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563dd124-d2fe-451e-b02b-3659394c5e59 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:03.623479 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c490d060-d5d1-4e88-9dcd-a82c0d27bd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:03.625532 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490d060-d5d1-4e88-9dcd-a82c0d27bd31 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:03.625797 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490d060-d5d1-4e88-9dcd-a82c0d27bd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:03.626103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490d060-d5d1-4e88-9dcd-a82c0d27bd31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:03.636570 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c490d060-d5d1-4e88-9dcd-a82c0d27bd31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:29:03.645723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-c490d060-d5d1-4e88-9dcd-a82c0d27bd31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:29:03.645723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:03.645723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:03.645723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:03.645723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:03.645723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:03.645723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:03.645723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:03.645723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:03.645723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:03.645723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:03.645723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:03.645723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:29:03.645723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:29:03.645723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:29:03.645723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:29:03.647044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:29:03.647044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:29:03.647044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:29:03.647044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:29:03.647044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:03.647044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:03.647044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:03.647044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:03.647044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:03.647044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:03.647044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:03.647044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:03.647044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:03.647044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:03.647044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:03.647044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:03.648079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:03.648079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:03.648079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:03.648079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:03.648079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:29:03.648079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:29:03.648079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:29:03.648079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:29:03.648079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:29:03.648079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:29:03.648079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 08:29:03.648079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:29:03.648079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 615/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Jul 18 08:29:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:03.653550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820fca20-a8a6-4001-bb00-60a03fb5eb2c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:03.701190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:03.758555 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820fca20-a8a6-4001-bb00-60a03fb5eb2c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:03.794342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7260f81-23c0-4898-82b5-dfd950bb898d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:03.795102 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7260f81-23c0-4898-82b5-dfd950bb898d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:03.795580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7260f81-23c0-4898-82b5-dfd950bb898d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:03.796290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7260f81-23c0-4898-82b5-dfd950bb898d None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:03.797079 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7260f81-23c0-4898-82b5-dfd950bb898d None admin] RBAC: Adding query filter params (name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:03.797387 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7260f81-23c0-4898-82b5-dfd950bb898d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:03.798756 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7260f81-23c0-4898-82b5-dfd950bb898d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:03.807539 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 621/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Jul 18 08:29:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:03.813650 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c490d060-d5d1-4e88-9dcd-a82c0d27bd31 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:03.904191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:03.921444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c490d060-d5d1-4e88-9dcd-a82c0d27bd31 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:03.955219 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e131040-0260-4149-9fdf-864d6504faf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:03.956057 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e131040-0260-4149-9fdf-864d6504faf2 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:03.956331 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e131040-0260-4149-9fdf-864d6504faf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:03.956607 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e131040-0260-4149-9fdf-864d6504faf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:03.966506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e131040-0260-4149-9fdf-864d6504faf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:03.968914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e131040-0260-4149-9fdf-864d6504faf2 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:03.986460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-4e131040-0260-4149-9fdf-864d6504faf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:29:03.986460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:03.986460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:03.986460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:03.986460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:03.986460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:03.986460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:03.986460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:03.986460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:03.986460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:03.986460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:03.986460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:03.986460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:29:03.986460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:29:03.986460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:29:03.987551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:29:03.987551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:03.987551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:03.987551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:03.987551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:03.987551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:03.987551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:03.987551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:03.987551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:03.987551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:03.987551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:03.987551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:03.987551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:03.987551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:03.987551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:03.987551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:03.987551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:03.988724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:29:03.988724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:29:03.988724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:29:03.988724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:29:03.988724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 616/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:03.993439 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7260f81-23c0-4898-82b5-dfd950bb898d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:04.094942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7260f81-23c0-4898-82b5-dfd950bb898d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:04.104319 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:04.129688 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5bb7c-98c3-4e13-82da-143f3f8bbac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:04.130348 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5bb7c-98c3-4e13-82da-143f3f8bbac9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:04.130632 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5bb7c-98c3-4e13-82da-143f3f8bbac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:04.130909 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5bb7c-98c3-4e13-82da-143f3f8bbac9 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:04.131630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5bb7c-98c3-4e13-82da-143f3f8bbac9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:04.131979 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5bb7c-98c3-4e13-82da-143f3f8bbac9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:04.133343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5bb7c-98c3-4e13-82da-143f3f8bbac9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:04.145688 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 622/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:04.151375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e131040-0260-4149-9fdf-864d6504faf2 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:04.252951 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e131040-0260-4149-9fdf-864d6504faf2 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:04.288228 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9b0b7-615f-4395-8433-d175b8bbc07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:04.289044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9b0b7-615f-4395-8433-d175b8bbc07a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:04.289321 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9b0b7-615f-4395-8433-d175b8bbc07a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:04.289590 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9b0b7-615f-4395-8433-d175b8bbc07a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:04.292805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:04.298033 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9b0b7-615f-4395-8433-d175b8bbc07a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:04.299499 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9b0b7-615f-4395-8433-d175b8bbc07a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:04.315943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-f8c9b0b7-615f-4395-8433-d175b8bbc07a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 08:29:04.315943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:04.315943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:04.315943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:04.315943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:04.315943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:04.315943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:04.315943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:04.315943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:04.315943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:04.315943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:04.315943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:04.315943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:29:04.315943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:29:04.315943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:29:04.315943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:29:04.321037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:04.321037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:04.321037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:29:04.321037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:29:04.321037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:29:04.321037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:29:04.321037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:04.321037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:04.321037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:04.321037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:04.321037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:04.321037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:04.321037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:04.321037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:04.321037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:04.321037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:04.321037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:04.321037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:04.322444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:04.322444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:04.322444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:29:04.322444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:29:04.322444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:29:04.322444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:29:04.322444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:29:04.322444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:29:04.322444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 08:29:04.322444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:29:04.322444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 617/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:29:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:04.323177 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5bb7c-98c3-4e13-82da-143f3f8bbac9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:04.424605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5bb7c-98c3-4e13-82da-143f3f8bbac9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:04.458986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6898c2-52d4-4b00-9b22-c2253abe83b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:04.459644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6898c2-52d4-4b00-9b22-c2253abe83b5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:04.459958 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6898c2-52d4-4b00-9b22-c2253abe83b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:04.460233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6898c2-52d4-4b00-9b22-c2253abe83b5 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:04.461071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6898c2-52d4-4b00-9b22-c2253abe83b5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:04.461381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6898c2-52d4-4b00-9b22-c2253abe83b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:04.462795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6898c2-52d4-4b00-9b22-c2253abe83b5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:04.479641 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 623/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Jul 18 08:29:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:04.486260 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9b0b7-615f-4395-8433-d175b8bbc07a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:04.587849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9b0b7-615f-4395-8433-d175b8bbc07a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:04.623478 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8717146-1764-4de1-adeb-f7c0f8a09a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:04.624786 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8717146-1764-4de1-adeb-f7c0f8a09a7d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:04.625048 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8717146-1764-4de1-adeb-f7c0f8a09a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:04.625324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8717146-1764-4de1-adeb-f7c0f8a09a7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:04.687788 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:04.693906 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8717146-1764-4de1-adeb-f7c0f8a09a7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:04.695370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8717146-1764-4de1-adeb-f7c0f8a09a7d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:04.798653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 618/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:29:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 08:29:04.804744 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6898c2-52d4-4b00-9b22-c2253abe83b5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:04.832188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:04.907848 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6898c2-52d4-4b00-9b22-c2253abe83b5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:04.942720 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c929f-cb8a-4610-a277-44d58eaa77e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:04.943421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c929f-cb8a-4610-a277-44d58eaa77e5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:04.943756 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c929f-cb8a-4610-a277-44d58eaa77e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:04.943994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c929f-cb8a-4610-a277-44d58eaa77e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:04.954050 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c929f-cb8a-4610-a277-44d58eaa77e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:04.955560 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c929f-cb8a-4610-a277-44d58eaa77e5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:04.974173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-2d4c929f-cb8a-4610-a277-44d58eaa77e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:29:04.974173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:04.974173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:04.974173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:04.974173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:04.974173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:04.974173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:04.974173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:04.974173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:04.974173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:04.974173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:04.974173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:04.974173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:29:04.974173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:29:04.974173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:29:04.975322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:29:04.975322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:04.975322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:04.975322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:04.975322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:04.975322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:04.975322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:04.975322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:04.975322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:04.975322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:04.975322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:04.975322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:04.975322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:04.975322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:04.975322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:04.975322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:04.975322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:04.976323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:29:04.976323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:29:04.976323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:29:04.976323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:29:04.976323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 624/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:29:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:04.981670 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8717146-1764-4de1-adeb-f7c0f8a09a7d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:05.079057 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:05.084833 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8717146-1764-4de1-adeb-f7c0f8a09a7d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:05.120829 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9659587-0033-4f4d-b6a5-10cf7ed2b143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:05.121490 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9659587-0033-4f4d-b6a5-10cf7ed2b143 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:05.121774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9659587-0033-4f4d-b6a5-10cf7ed2b143 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:05.122052 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9659587-0033-4f4d-b6a5-10cf7ed2b143 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:05.122778 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9659587-0033-4f4d-b6a5-10cf7ed2b143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:05.123086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9659587-0033-4f4d-b6a5-10cf7ed2b143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:05.124698 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9659587-0033-4f4d-b6a5-10cf7ed2b143 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:05.136297 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 619/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:05.142581 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c929f-cb8a-4610-a277-44d58eaa77e5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:05.224711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:05.247210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c929f-cb8a-4610-a277-44d58eaa77e5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:05.285067 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ceaac-fe37-472a-a21a-31799cc05c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:05.285883 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ceaac-fe37-472a-a21a-31799cc05c78 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:05.286197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ceaac-fe37-472a-a21a-31799cc05c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:05.286477 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ceaac-fe37-472a-a21a-31799cc05c78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:05.293656 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ceaac-fe37-472a-a21a-31799cc05c78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:05.295111 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ceaac-fe37-472a-a21a-31799cc05c78 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:05.309396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-635ceaac-fe37-472a-a21a-31799cc05c78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:29:05.309396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:05.309396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:05.309396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:05.309396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:05.309396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:05.309396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:05.309396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:05.309396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:05.309396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:05.309396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:05.309396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:05.309396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:29:05.309396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:29:05.309396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:29:05.310964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:29:05.310964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:05.310964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:05.310964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:05.310964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:05.310964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:05.310964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:05.310964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:05.310964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:05.310964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:05.310964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:05.310964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:05.310964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:05.310964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:05.310964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:05.310964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:05.310964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:05.313306 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:29:05.313306 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:29:05.313306 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:29:05.313306 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:29:05.313306 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:29:05.313306 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:29:05.313306 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:29:05.313306 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:29:05.313306 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 625/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:29:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:05.316699 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9659587-0033-4f4d-b6a5-10cf7ed2b143 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:05.420869 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9659587-0033-4f4d-b6a5-10cf7ed2b143 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:05.460556 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15bf18-9045-4b42-a44a-d373ce26f6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:05.461217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15bf18-9045-4b42-a44a-d373ce26f6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:05.461496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15bf18-9045-4b42-a44a-d373ce26f6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:05.461765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15bf18-9045-4b42-a44a-d373ce26f6c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:05.462540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15bf18-9045-4b42-a44a-d373ce26f6c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:05.462859 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15bf18-9045-4b42-a44a-d373ce26f6c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:05.464329 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15bf18-9045-4b42-a44a-d373ce26f6c0 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:05.477963 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 620/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:29:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:05.484215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ceaac-fe37-472a-a21a-31799cc05c78 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:05.606080 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ceaac-fe37-472a-a21a-31799cc05c78 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:05.640903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a680e-6545-4be8-b65d-f33d9de2b3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:05.641563 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a680e-6545-4be8-b65d-f33d9de2b3dd None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:05.641849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a680e-6545-4be8-b65d-f33d9de2b3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:05.642127 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a680e-6545-4be8-b65d-f33d9de2b3dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:05.642961 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a680e-6545-4be8-b65d-f33d9de2b3dd None admin] RBAC: Adding query filter params (role.id=9fc4c3dcb90945ebb4eb5566e62e7833, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=d606fd7ef269460a8110223932f02d19) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:05.643276 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a680e-6545-4be8-b65d-f33d9de2b3dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:05.644769 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a680e-6545-4be8-b65d-f33d9de2b3dd None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:05.650848 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 626/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:29:05 2022] GET /identity/v3/role_assignments?user.id=d606fd7ef269460a8110223932f02d19&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=9fc4c3dcb90945ebb4eb5566e62e7833 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:29:07.788038 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab8fb2-f5c9-44af-961b-2e34eafb0355 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:07.788600 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab8fb2-f5c9-44af-961b-2e34eafb0355 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:07.788600 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab8fb2-f5c9-44af-961b-2e34eafb0355 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:07.790044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 621/1247] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:29:07.795391 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83518b80-5428-463d-b428-ed1711d8c6ab None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:07.795859 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83518b80-5428-463d-b428-ed1711d8c6ab None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:07.796023 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83518b80-5428-463d-b428-ed1711d8c6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:07.801315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:07.814101 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-83518b80-5428-463d-b428-ed1711d8c6ab None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:29:07.816967 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-83518b80-5428-463d-b428-ed1711d8c6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:29:07.882443 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 627/1248] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:29:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:29:07.906429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab8fb2-f5c9-44af-961b-2e34eafb0355 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:07.924491 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:08.007661 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab8fb2-f5c9-44af-961b-2e34eafb0355 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:08.041721 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f6761-dec8-4f2a-93d5-ed36e993f9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:08.042434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f6761-dec8-4f2a-93d5-ed36e993f9db None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:08.042713 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f6761-dec8-4f2a-93d5-ed36e993f9db None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:08.042998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f6761-dec8-4f2a-93d5-ed36e993f9db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:08.053392 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f6761-dec8-4f2a-93d5-ed36e993f9db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:29:08.062029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-576f6761-dec8-4f2a-93d5-ed36e993f9db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:29:08.062029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:08.062029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:08.062029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:08.062029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:08.062029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:08.062029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:08.062029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:08.062029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:08.062029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:08.062029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:08.062029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:08.062029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:29:08.062029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:29:08.062029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:29:08.062029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:29:08.066967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:29:08.066967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:29:08.066967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:29:08.066967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:29:08.066967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:08.066967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:08.066967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:08.066967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:08.066967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:08.066967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:08.066967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:08.066967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:08.066967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:08.066967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:08.066967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:08.066967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:08.068046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:08.068046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:08.068046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:08.068046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:08.068046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:29:08.068046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:29:08.068046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:29:08.068046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:29:08.068046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:29:08.068046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:29:08.068046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:29:08.068046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:29:08.068046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 622/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:29:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:08.069489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83518b80-5428-463d-b428-ed1711d8c6ab None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:08.172897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83518b80-5428-463d-b428-ed1711d8c6ab None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:08.203260 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:08.208202 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ca305-5499-4d25-aab9-ed9ef075e302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:08.208866 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ca305-5499-4d25-aab9-ed9ef075e302 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:08.209150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ca305-5499-4d25-aab9-ed9ef075e302 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:08.209426 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ca305-5499-4d25-aab9-ed9ef075e302 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:08.210172 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ca305-5499-4d25-aab9-ed9ef075e302 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:08.210478 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ca305-5499-4d25-aab9-ed9ef075e302 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:08.211921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ca305-5499-4d25-aab9-ed9ef075e302 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:08.220148 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 628/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:29:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:29:08.225883 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f6761-dec8-4f2a-93d5-ed36e993f9db None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:08.308038 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:08.335821 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f6761-dec8-4f2a-93d5-ed36e993f9db None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:08.373110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95df2b-9801-43d7-b092-8152b8488494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:08.374520 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95df2b-9801-43d7-b092-8152b8488494 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:08.374801 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95df2b-9801-43d7-b092-8152b8488494 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:08.375075 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95df2b-9801-43d7-b092-8152b8488494 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:08.386155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95df2b-9801-43d7-b092-8152b8488494 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:08.387635 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95df2b-9801-43d7-b092-8152b8488494 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:08.406680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-bf95df2b-9801-43d7-b092-8152b8488494 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:29:08.406680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:08.406680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:08.406680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:08.406680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:08.406680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:08.406680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:08.406680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:08.406680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:08.406680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:08.406680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:08.406680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:08.406680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:29:08.406680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:29:08.406680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:29:08.414320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:29:08.414320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:08.414320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:08.414320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:08.414320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:08.414320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:08.414320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:08.414320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:08.414320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:08.414320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:08.414320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:08.414320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:08.414320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:08.414320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:08.414320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:08.414320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:08.414320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:08.415425 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:29:08.415425 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:29:08.415425 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:29:08.415425 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:29:08.415425 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 623/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:29:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:08.415806 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ca305-5499-4d25-aab9-ed9ef075e302 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:08.519916 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ca305-5499-4d25-aab9-ed9ef075e302 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:08.556022 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2af4d4-f938-46ad-95b8-bf65af196526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:08.556692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2af4d4-f938-46ad-95b8-bf65af196526 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:08.556979 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2af4d4-f938-46ad-95b8-bf65af196526 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:08.557349 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2af4d4-f938-46ad-95b8-bf65af196526 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:08.558632 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2af4d4-f938-46ad-95b8-bf65af196526 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:08.558884 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2af4d4-f938-46ad-95b8-bf65af196526 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:08.560282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2af4d4-f938-46ad-95b8-bf65af196526 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:08.572337 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 629/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:08.578267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95df2b-9801-43d7-b092-8152b8488494 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:08.679435 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95df2b-9801-43d7-b092-8152b8488494 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:08.705969 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:08.714174 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788917c-d2e9-4543-8fa1-33c9e7ce4b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:08.714963 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788917c-d2e9-4543-8fa1-33c9e7ce4b53 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:08.715247 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788917c-d2e9-4543-8fa1-33c9e7ce4b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:08.715524 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788917c-d2e9-4543-8fa1-33c9e7ce4b53 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:08.724141 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788917c-d2e9-4543-8fa1-33c9e7ce4b53 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:08.725608 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788917c-d2e9-4543-8fa1-33c9e7ce4b53 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:08.740795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-3788917c-d2e9-4543-8fa1-33c9e7ce4b53 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 08:29:08.740795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:08.740795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:08.740795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:08.740795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:08.740795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:08.740795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:08.740795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:08.740795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:08.740795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:08.740795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:08.740795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:08.740795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:29:08.740795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:29:08.740795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:29:08.740795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:29:08.742000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:08.742000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:08.742000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:29:08.742000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:29:08.742000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:29:08.742000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:29:08.742000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:08.742000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:08.742000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:08.742000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:08.742000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:08.742000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:08.742000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:08.742000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:08.742000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:08.742000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:08.742000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:08.742000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:08.743206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:08.743206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:08.743206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:29:08.743206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:29:08.743206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:29:08.743206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:29:08.743206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:29:08.743206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:29:08.743206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 08:29:08.743206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:29:08.743206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 624/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:29:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:08.747910 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2af4d4-f938-46ad-95b8-bf65af196526 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:08.805874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:08.852138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2af4d4-f938-46ad-95b8-bf65af196526 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:08.890432 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229b816-c078-4006-a9f1-781b580ece25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:08.891141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229b816-c078-4006-a9f1-781b580ece25 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:08.891422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229b816-c078-4006-a9f1-781b580ece25 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:08.891796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229b816-c078-4006-a9f1-781b580ece25 None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:08.892589 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a229b816-c078-4006-a9f1-781b580ece25 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:08.892903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a229b816-c078-4006-a9f1-781b580ece25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:08.894335 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a229b816-c078-4006-a9f1-781b580ece25 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:08.912713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 630/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Jul 18 08:29:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:08.918688 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788917c-d2e9-4543-8fa1-33c9e7ce4b53 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:09.022017 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788917c-d2e9-4543-8fa1-33c9e7ce4b53 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:09.056257 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f931c8-8e20-4b1f-a33b-ebc8f9e3515d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:09.056907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f931c8-8e20-4b1f-a33b-ebc8f9e3515d None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:09.057190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f931c8-8e20-4b1f-a33b-ebc8f9e3515d None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:09.057496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f931c8-8e20-4b1f-a33b-ebc8f9e3515d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:09.114073 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:09.126221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f931c8-8e20-4b1f-a33b-ebc8f9e3515d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:09.127699 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f931c8-8e20-4b1f-a33b-ebc8f9e3515d None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:09.230170 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 625/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:29:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 08:29:09.236063 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229b816-c078-4006-a9f1-781b580ece25 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:09.338426 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229b816-c078-4006-a9f1-781b580ece25 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:09.352579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:09.377973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7942f7b-e60a-4c0f-bcd9-ee6562d67bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:09.378681 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7942f7b-e60a-4c0f-bcd9-ee6562d67bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:09.378970 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7942f7b-e60a-4c0f-bcd9-ee6562d67bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:09.379253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7942f7b-e60a-4c0f-bcd9-ee6562d67bc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:09.390264 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7942f7b-e60a-4c0f-bcd9-ee6562d67bc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:09.391910 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7942f7b-e60a-4c0f-bcd9-ee6562d67bc9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:09.410421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-c7942f7b-e60a-4c0f-bcd9-ee6562d67bc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:29:09.410421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:09.410421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:09.410421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:09.410421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:09.410421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:09.410421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:09.410421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:09.410421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:09.410421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:09.410421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:09.410421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:09.410421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:29:09.410421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:29:09.410421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:29:09.411643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:29:09.411643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:09.411643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:09.411643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:09.411643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:09.411643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:09.411643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:09.411643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:09.411643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:09.411643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:09.411643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:09.411643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:09.411643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:09.411643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:09.411643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:09.411643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:09.411643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:09.418279 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:29:09.418279 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:29:09.418279 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:29:09.418279 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:29:09.418279 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 631/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:09.418568 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f931c8-8e20-4b1f-a33b-ebc8f9e3515d None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:09.511895 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:09.524514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f931c8-8e20-4b1f-a33b-ebc8f9e3515d None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:09.559423 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ccf877-9edb-4bfa-a354-f2dfb14f9a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:09.560128 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccf877-9edb-4bfa-a354-f2dfb14f9a6a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:09.560407 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccf877-9edb-4bfa-a354-f2dfb14f9a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:09.561225 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccf877-9edb-4bfa-a354-f2dfb14f9a6a None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:09.562058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ccf877-9edb-4bfa-a354-f2dfb14f9a6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:09.562321 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ccf877-9edb-4bfa-a354-f2dfb14f9a6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:09.563699 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ccf877-9edb-4bfa-a354-f2dfb14f9a6a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:09.581649 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 626/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:09.587647 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7942f7b-e60a-4c0f-bcd9-ee6562d67bc9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:09.694952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7942f7b-e60a-4c0f-bcd9-ee6562d67bc9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:09.729970 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7aa74-1994-4371-b989-3ed17452724b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:09.730659 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7aa74-1994-4371-b989-3ed17452724b None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:09.730946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7aa74-1994-4371-b989-3ed17452724b None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:09.731224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7aa74-1994-4371-b989-3ed17452724b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:09.738860 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7aa74-1994-4371-b989-3ed17452724b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:09.740342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7aa74-1994-4371-b989-3ed17452724b None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:09.754707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-2cf7aa74-1994-4371-b989-3ed17452724b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:29:09.754707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:09.754707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:09.754707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:09.754707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:09.754707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:09.754707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:09.754707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:09.754707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:09.754707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:09.754707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:09.754707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:09.754707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:29:09.754707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:29:09.754707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:29:09.755752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:29:09.755752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:09.755752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:09.755752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:09.755752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:09.755752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:09.755752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:09.755752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:09.755752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:09.755752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:09.755752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:09.755752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:09.755752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:09.755752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:09.755752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:09.755752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:09.755752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:09.756676 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:29:09.756676 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:29:09.756676 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:29:09.756676 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:29:09.756676 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:29:09.756676 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:29:09.756676 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:29:09.756676 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:29:09.756676 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 632/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:29:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:09.762131 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ccf877-9edb-4bfa-a354-f2dfb14f9a6a None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:09.864148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ccf877-9edb-4bfa-a354-f2dfb14f9a6a None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:09.898684 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd580232-25ea-44ca-ae9e-4558cb6b87a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:09.899337 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd580232-25ea-44ca-ae9e-4558cb6b87a0 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:09.899621 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd580232-25ea-44ca-ae9e-4558cb6b87a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:09.899938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd580232-25ea-44ca-ae9e-4558cb6b87a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:09.900724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd580232-25ea-44ca-ae9e-4558cb6b87a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:09.901022 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd580232-25ea-44ca-ae9e-4558cb6b87a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:09.902455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd580232-25ea-44ca-ae9e-4558cb6b87a0 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:09.915659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 627/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:29:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:09.923402 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7aa74-1994-4371-b989-3ed17452724b None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:09.955507 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:10.032799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7aa74-1994-4371-b989-3ed17452724b None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:10.067734 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979cc144-cf1c-4b52-a089-da92e2162845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:10.068399 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cc144-cf1c-4b52-a089-da92e2162845 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:10.068770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cc144-cf1c-4b52-a089-da92e2162845 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:10.069062 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cc144-cf1c-4b52-a089-da92e2162845 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:10.069908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979cc144-cf1c-4b52-a089-da92e2162845 None admin] RBAC: Adding query filter params (role.id=467d49b8a0d3435aaba58464cc8ef85c, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=d606fd7ef269460a8110223932f02d19) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:10.070223 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979cc144-cf1c-4b52-a089-da92e2162845 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:10.071685 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979cc144-cf1c-4b52-a089-da92e2162845 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:10.077777 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 633/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:29:09 2022] GET /identity/v3/role_assignments?user.id=d606fd7ef269460a8110223932f02d19&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=467d49b8a0d3435aaba58464cc8ef85c => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:12.126586 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f40e5-dc50-4fc7-af82-e4fd5edb2b79 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:12.127088 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f40e5-dc50-4fc7-af82-e4fd5edb2b79 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:12.127088 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f40e5-dc50-4fc7-af82-e4fd5edb2b79 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:12.128524 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 628/1261] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:29:12.133849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90d0a0-cfed-4b3d-9fe3-61dd06b39366 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:12.134311 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90d0a0-cfed-4b3d-9fe3-61dd06b39366 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:12.134438 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90d0a0-cfed-4b3d-9fe3-61dd06b39366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:12.151915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3b90d0a0-cfed-4b3d-9fe3-61dd06b39366 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:29:12.153122 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3b90d0a0-cfed-4b3d-9fe3-61dd06b39366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:29:12.221293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 634/1262] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:29:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:29:12.243552 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f40e5-dc50-4fc7-af82-e4fd5edb2b79 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:12.255591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:12.347080 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f40e5-dc50-4fc7-af82-e4fd5edb2b79 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:12.381063 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6323fd-13fe-4255-9614-7c77bd70c821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:12.381802 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6323fd-13fe-4255-9614-7c77bd70c821 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:12.382090 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6323fd-13fe-4255-9614-7c77bd70c821 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:12.382366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6323fd-13fe-4255-9614-7c77bd70c821 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:12.393329 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6323fd-13fe-4255-9614-7c77bd70c821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:29:12.400984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-cb6323fd-13fe-4255-9614-7c77bd70c821 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:29:12.400984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:12.400984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:12.400984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:12.400984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:12.400984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:12.400984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:12.400984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:12.400984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:12.400984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:12.400984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:12.400984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:12.400984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:29:12.400984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:29:12.400984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:29:12.400984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:29:12.402138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:29:12.402138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:29:12.402138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:29:12.402138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:29:12.402138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:12.402138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:12.402138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:12.402138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:12.402138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:12.402138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:12.402138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:12.402138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:12.402138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:12.402138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:12.402138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:12.402138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:12.403359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:12.403359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:12.403359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:12.403359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:12.403359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:29:12.403359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:29:12.403359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:29:12.403359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:29:12.403359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:29:12.403359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:29:12.403359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:29:12.403359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:29:12.403359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 629/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:29:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:12.408493 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90d0a0-cfed-4b3d-9fe3-61dd06b39366 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:12.427614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:12.509114 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90d0a0-cfed-4b3d-9fe3-61dd06b39366 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:12.545901 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328e266-58ef-4f71-a441-7867c182a679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:12.628670 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328e266-58ef-4f71-a441-7867c182a679 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:12.628670 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328e266-58ef-4f71-a441-7867c182a679 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:12.628670 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328e266-58ef-4f71-a441-7867c182a679 None admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:12.628670 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328e266-58ef-4f71-a441-7867c182a679 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:12.628670 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328e266-58ef-4f71-a441-7867c182a679 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:12.628670 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328e266-58ef-4f71-a441-7867c182a679 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:12.629233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 635/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:29:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:12.629325 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6323fd-13fe-4255-9614-7c77bd70c821 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:12.637432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:12.672861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6323fd-13fe-4255-9614-7c77bd70c821 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:12.706934 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba78a10-0ebb-4434-b04a-18a8a96228f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:12.707747 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba78a10-0ebb-4434-b04a-18a8a96228f6 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:12.708030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba78a10-0ebb-4434-b04a-18a8a96228f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:12.708304 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba78a10-0ebb-4434-b04a-18a8a96228f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:12.719874 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba78a10-0ebb-4434-b04a-18a8a96228f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:12.721516 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba78a10-0ebb-4434-b04a-18a8a96228f6 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:12.740901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-3ba78a10-0ebb-4434-b04a-18a8a96228f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:29:12.740901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:12.740901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:12.740901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:12.740901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:12.740901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:12.740901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:12.740901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:12.740901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:12.740901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:12.740901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:12.740901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:12.740901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:29:12.740901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:29:12.740901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:29:12.742144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:29:12.742144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:12.742144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:12.742144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:12.742144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:12.742144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:12.742144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:12.742144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:12.742144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:12.742144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:12.742144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:12.742144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:12.742144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:12.742144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:12.742144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:12.742144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:12.742144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:12.743469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:29:12.743469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:29:12.743469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:29:12.743469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:29:12.743469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 630/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:29:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:12.748232 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328e266-58ef-4f71-a441-7867c182a679 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:12.832837 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:12.852240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328e266-58ef-4f71-a441-7867c182a679 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:12.886235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f51502-a733-468b-9137-aa5672cf0840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:12.886975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f51502-a733-468b-9137-aa5672cf0840 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:12.887264 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f51502-a733-468b-9137-aa5672cf0840 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:12.887591 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f51502-a733-468b-9137-aa5672cf0840 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:12.888320 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f51502-a733-468b-9137-aa5672cf0840 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:12.888636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f51502-a733-468b-9137-aa5672cf0840 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:12.890634 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f51502-a733-468b-9137-aa5672cf0840 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:12.902698 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 636/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:12.909529 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba78a10-0ebb-4434-b04a-18a8a96228f6 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:13.009877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba78a10-0ebb-4434-b04a-18a8a96228f6 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:13.031141 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:13.044734 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa0205-bfc9-4ae5-8a36-a06eb26e312e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:13.046461 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa0205-bfc9-4ae5-8a36-a06eb26e312e None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:13.046745 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa0205-bfc9-4ae5-8a36-a06eb26e312e None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:13.047023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa0205-bfc9-4ae5-8a36-a06eb26e312e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:13.054537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa0205-bfc9-4ae5-8a36-a06eb26e312e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:13.056016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa0205-bfc9-4ae5-8a36-a06eb26e312e None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:13.070379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-59fa0205-bfc9-4ae5-8a36-a06eb26e312e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 08:29:13.070379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:13.070379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:13.070379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:13.070379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:13.070379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:13.070379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:13.070379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:13.070379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:13.070379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:13.070379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:13.070379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:13.070379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:29:13.070379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:29:13.070379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:29:13.070379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:29:13.071547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:13.071547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:13.071547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:29:13.071547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:29:13.071547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:29:13.071547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:29:13.071547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:13.071547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:13.071547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:13.071547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:13.071547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:13.071547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:13.071547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:13.071547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:13.071547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:13.071547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:13.071547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:13.071547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:13.072772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:13.072772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:13.072772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:29:13.072772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:29:13.072772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:29:13.072772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:29:13.072772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:29:13.072772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:29:13.072772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 08:29:13.072772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:29:13.072772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 631/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:29:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:13.077843 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f51502-a733-468b-9137-aa5672cf0840 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:13.181283 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f51502-a733-468b-9137-aa5672cf0840 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:13.215198 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab3574-c103-4842-8647-ac81ea21e30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:13.215901 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab3574-c103-4842-8647-ac81ea21e30d None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:13.216186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab3574-c103-4842-8647-ac81ea21e30d None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:13.216464 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab3574-c103-4842-8647-ac81ea21e30d None admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:13.217960 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab3574-c103-4842-8647-ac81ea21e30d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:13.218486 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab3574-c103-4842-8647-ac81ea21e30d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:13.219708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab3574-c103-4842-8647-ac81ea21e30d None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:13.229163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:13.238470 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 637/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Jul 18 08:29:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:13.244599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa0205-bfc9-4ae5-8a36-a06eb26e312e None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:13.347513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa0205-bfc9-4ae5-8a36-a06eb26e312e None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:13.386449 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66bbc6-1103-43c8-96ca-36c1e4bfe0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:13.387241 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66bbc6-1103-43c8-96ca-36c1e4bfe0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:13.387525 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66bbc6-1103-43c8-96ca-36c1e4bfe0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:13.387830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66bbc6-1103-43c8-96ca-36c1e4bfe0c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:13.399692 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66bbc6-1103-43c8-96ca-36c1e4bfe0c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:13.401232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66bbc6-1103-43c8-96ca-36c1e4bfe0c7 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:13.426143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-eb66bbc6-1103-43c8-96ca-36c1e4bfe0c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:29:13.426143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:13.426143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:13.426143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:13.426143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:13.426143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:13.426143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:13.426143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:13.426143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:13.426143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:13.426143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:13.426143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:13.426143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:29:13.426143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:29:13.426143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:29:13.427278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:29:13.427278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:13.427278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:13.427278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:13.427278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:13.427278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:13.427278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:13.427278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:13.427278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:13.427278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:13.427278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:13.427278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:13.427278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:13.427278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:13.427278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:13.427278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:13.427278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:13.428434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:29:13.428434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:29:13.428434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:29:13.428434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:29:13.428434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 632/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:29:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:13.433633 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab3574-c103-4842-8647-ac81ea21e30d None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:13.540164 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab3574-c103-4842-8647-ac81ea21e30d None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:13.574945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d10d80-1250-42d3-9153-c0ec377eceea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:13.575660 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d10d80-1250-42d3-9153-c0ec377eceea None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:13.575981 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d10d80-1250-42d3-9153-c0ec377eceea None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:13.576933 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d10d80-1250-42d3-9153-c0ec377eceea None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:13.577925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d10d80-1250-42d3-9153-c0ec377eceea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:13.578240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d10d80-1250-42d3-9153-c0ec377eceea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:13.579717 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d10d80-1250-42d3-9153-c0ec377eceea None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:13.592467 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 638/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:13.598560 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66bbc6-1103-43c8-96ca-36c1e4bfe0c7 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:13.631172 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:13.700346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66bbc6-1103-43c8-96ca-36c1e4bfe0c7 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:13.735880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1aed16-5c36-4a76-ab3f-d85f0971fbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:13.744167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1aed16-5c36-4a76-ab3f-d85f0971fbf4 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:13.744167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1aed16-5c36-4a76-ab3f-d85f0971fbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:13.744167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1aed16-5c36-4a76-ab3f-d85f0971fbf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:13.744490 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1aed16-5c36-4a76-ab3f-d85f0971fbf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:13.746071 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1aed16-5c36-4a76-ab3f-d85f0971fbf4 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:13.760885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-8c1aed16-5c36-4a76-ab3f-d85f0971fbf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:29:13.760885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:13.760885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:13.760885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:13.760885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:13.760885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:13.760885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:13.760885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:13.760885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:13.760885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:13.760885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:13.760885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:13.760885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:29:13.760885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:29:13.760885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:29:13.762004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:29:13.762004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:13.762004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:13.762004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:13.762004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:13.762004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:13.762004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:13.762004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:13.762004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:13.762004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:13.762004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:13.762004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:13.762004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:13.762004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:13.762004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:13.762004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:13.762004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:13.763279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:29:13.763279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:29:13.763279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:29:13.763279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:29:13.763279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:29:13.763279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:29:13.763279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:29:13.763279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:29:13.763279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 633/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:29:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 08:29:13.768680 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d10d80-1250-42d3-9153-c0ec377eceea None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:13.832578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:13.875009 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d10d80-1250-42d3-9153-c0ec377eceea None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:13.908972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfe367-3011-4f3a-a77e-fd3fe9ee57ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:13.909628 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfe367-3011-4f3a-a77e-fd3fe9ee57ef None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:13.909910 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfe367-3011-4f3a-a77e-fd3fe9ee57ef None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:13.910186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfe367-3011-4f3a-a77e-fd3fe9ee57ef None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:13.910973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfe367-3011-4f3a-a77e-fd3fe9ee57ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:13.911282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfe367-3011-4f3a-a77e-fd3fe9ee57ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:13.912756 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfe367-3011-4f3a-a77e-fd3fe9ee57ef None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:13.924821 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 639/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:29:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:13.930921 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1aed16-5c36-4a76-ab3f-d85f0971fbf4 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:14.026308 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:14.035016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1aed16-5c36-4a76-ab3f-d85f0971fbf4 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:14.070867 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab3b63-2d0f-489a-8713-032d4d523607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:14.071722 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab3b63-2d0f-489a-8713-032d4d523607 None admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:14.072084 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab3b63-2d0f-489a-8713-032d4d523607 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:14.072302 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab3b63-2d0f-489a-8713-032d4d523607 None admin] PATH_INFO: `/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/d606fd7ef269460a8110223932f02d19/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:14.074270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab3b63-2d0f-489a-8713-032d4d523607 None admin] RBAC: Authorizing `identity:create_grant(project_id=35c846931fd443fbb83a6fbe0aeaa866, user_id=d606fd7ef269460a8110223932f02d19, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:14.075718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab3b63-2d0f-489a-8713-032d4d523607 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:14.198548 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 634/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Jul 18 08:29:13 2022] PUT /identity/v3/projects/35c846931fd443fbb83a6fbe0aeaa866/users/d606fd7ef269460a8110223932f02d19/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:29:16.269972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886c56a-aa90-4fe7-8bc4-28e20aea3a7b None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:16.270615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886c56a-aa90-4fe7-8bc4-28e20aea3a7b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:16.270615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886c56a-aa90-4fe7-8bc4-28e20aea3a7b None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:16.271933 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 640/1274] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:29:16.277357 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cce45-fb0b-48b5-ba40-e52e6e1c2e3f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:16.277787 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cce45-fb0b-48b5-ba40-e52e6e1c2e3f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:16.277939 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cce45-fb0b-48b5-ba40-e52e6e1c2e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:16.296474 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-996cce45-fb0b-48b5-ba40-e52e6e1c2e3f None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:29:16.297787 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-996cce45-fb0b-48b5-ba40-e52e6e1c2e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:29:16.367292 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 635/1275] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:29:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 08:29:16.391330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886c56a-aa90-4fe7-8bc4-28e20aea3a7b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:16.493421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886c56a-aa90-4fe7-8bc4-28e20aea3a7b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:16.512052 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:16.529300 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba57b1-3d9c-4269-b73a-1464eedbd93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:16.530810 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba57b1-3d9c-4269-b73a-1464eedbd93a None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:16.531115 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba57b1-3d9c-4269-b73a-1464eedbd93a None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:16.531431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba57b1-3d9c-4269-b73a-1464eedbd93a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:16.542060 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba57b1-3d9c-4269-b73a-1464eedbd93a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 08:29:16.550136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-ceba57b1-3d9c-4269-b73a-1464eedbd93a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:29:16.550136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:16.550136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:16.550136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:16.550136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:16.550136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:16.550136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:16.550136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:16.550136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:16.550136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:16.550136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:16.550136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:16.550136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 08:29:16.550136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 08:29:16.550136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 08:29:16.550136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 08:29:16.551474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 08:29:16.551474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 08:29:16.551474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 08:29:16.551474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 08:29:16.551474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:16.551474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:16.551474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:16.551474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:16.551474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:16.551474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:16.551474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:16.551474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:16.551474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:16.551474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:16.551474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:16.551474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:16.552993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:16.552993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:16.552993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:16.552993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:16.552993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 08:29:16.552993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 08:29:16.552993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 08:29:16.552993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 08:29:16.552993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 08:29:16.552993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 08:29:16.552993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 08:29:16.552993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:29:16.552993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 641/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Jul 18 08:29:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:16.557676 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cce45-fb0b-48b5-ba40-e52e6e1c2e3f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:16.646998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:16.665338 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cce45-fb0b-48b5-ba40-e52e6e1c2e3f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:16.699049 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f5fa8-a257-4dc7-9637-0541827acc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:16.699739 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f5fa8-a257-4dc7-9637-0541827acc56 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:16.700028 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f5fa8-a257-4dc7-9637-0541827acc56 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:16.700303 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f5fa8-a257-4dc7-9637-0541827acc56 None admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:16.701041 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f5fa8-a257-4dc7-9637-0541827acc56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:16.701863 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f5fa8-a257-4dc7-9637-0541827acc56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:16.703375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f5fa8-a257-4dc7-9637-0541827acc56 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:16.711478 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 636/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Jul 18 08:29:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:16.718080 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba57b1-3d9c-4269-b73a-1464eedbd93a None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:16.825691 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba57b1-3d9c-4269-b73a-1464eedbd93a None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:16.865135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec22f9-2d4c-4f7b-a043-16ec444e3d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:16.865963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec22f9-2d4c-4f7b-a043-16ec444e3d18 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:16.866244 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec22f9-2d4c-4f7b-a043-16ec444e3d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:16.866513 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec22f9-2d4c-4f7b-a043-16ec444e3d18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:16.878311 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec22f9-2d4c-4f7b-a043-16ec444e3d18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:16.881624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec22f9-2d4c-4f7b-a043-16ec444e3d18 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:16.898814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-caec22f9-2d4c-4f7b-a043-16ec444e3d18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:29:16.898814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:16.898814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:16.898814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:16.898814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:16.898814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:16.898814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:16.898814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:16.898814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:16.898814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:16.898814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:16.898814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:16.898814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:29:16.898814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:29:16.898814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:29:16.899991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:29:16.899991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:16.899991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:16.899991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:16.899991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:16.899991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:16.899991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:16.899991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:16.899991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:16.899991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:16.899991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:16.899991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:16.899991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:16.899991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:16.899991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:16.899991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:16.899991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:16.901110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:29:16.901110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:29:16.901110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:29:16.901110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:29:16.901110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 642/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:29:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:16.906089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f5fa8-a257-4dc7-9637-0541827acc56 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:17.009128 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f5fa8-a257-4dc7-9637-0541827acc56 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:17.046752 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308941a4-98ad-4025-9566-b02ca3231f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:17.047418 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308941a4-98ad-4025-9566-b02ca3231f26 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:17.051738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308941a4-98ad-4025-9566-b02ca3231f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:17.051738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308941a4-98ad-4025-9566-b02ca3231f26 None admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:17.051738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308941a4-98ad-4025-9566-b02ca3231f26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:17.051738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308941a4-98ad-4025-9566-b02ca3231f26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:17.051738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308941a4-98ad-4025-9566-b02ca3231f26 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:17.063008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 637/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:17.068799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec22f9-2d4c-4f7b-a043-16ec444e3d18 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:17.100834 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:17.170973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec22f9-2d4c-4f7b-a043-16ec444e3d18 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:17.205513 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327b1e1-690c-49af-a487-23919744ddea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:17.206411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327b1e1-690c-49af-a487-23919744ddea None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:17.206669 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327b1e1-690c-49af-a487-23919744ddea None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:17.206884 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327b1e1-690c-49af-a487-23919744ddea None admin] PATH_INFO: `/v3/users/placement` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:17.213808 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327b1e1-690c-49af-a487-23919744ddea None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:17.215237 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327b1e1-690c-49af-a487-23919744ddea None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:17.230888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-e327b1e1-690c-49af-a487-23919744ddea None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 08:29:17.230888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:17.230888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:17.230888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:17.230888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:17.230888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:17.230888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:17.230888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:17.230888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:17.230888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:17.230888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:17.230888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:17.230888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 08:29:17.230888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 08:29:17.230888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 08:29:17.230888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 08:29:17.231963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:17.231963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:17.231963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 08:29:17.231963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:29:17.231963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 08:29:17.231963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 08:29:17.231963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:17.231963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:17.231963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:17.231963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:17.231963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:17.231963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:17.231963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:17.231963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:17.231963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:17.231963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:17.231963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:17.231963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:17.233126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:17.233126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:17.233126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 08:29:17.233126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 08:29:17.233126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 08:29:17.233126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 08:29:17.233126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 08:29:17.233126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 08:29:17.233126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 08:29:17.233126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:29:17.233126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 643/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:29:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:17.238270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308941a4-98ad-4025-9566-b02ca3231f26 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:17.245008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:17.339982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308941a4-98ad-4025-9566-b02ca3231f26 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:17.377290 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413284ac-89a7-4b70-99bc-c7d7b688dbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:17.377898 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413284ac-89a7-4b70-99bc-c7d7b688dbdb None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:17.378177 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413284ac-89a7-4b70-99bc-c7d7b688dbdb None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:17.378446 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413284ac-89a7-4b70-99bc-c7d7b688dbdb None admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:17.379276 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413284ac-89a7-4b70-99bc-c7d7b688dbdb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:17.379578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413284ac-89a7-4b70-99bc-c7d7b688dbdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:17.381041 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413284ac-89a7-4b70-99bc-c7d7b688dbdb None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:17.398408 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 638/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Jul 18 08:29:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:17.404326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327b1e1-690c-49af-a487-23919744ddea None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:17.497821 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:17.508680 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327b1e1-690c-49af-a487-23919744ddea None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:17.544146 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09b651-42f3-4520-9c05-14fa6a9add62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:17.544887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09b651-42f3-4520-9c05-14fa6a9add62 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:17.545728 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09b651-42f3-4520-9c05-14fa6a9add62 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:17.546040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09b651-42f3-4520-9c05-14fa6a9add62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:17.616086 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d09b651-42f3-4520-9c05-14fa6a9add62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:17.617638 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d09b651-42f3-4520-9c05-14fa6a9add62 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:17.709210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:17.719411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 644/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Jul 18 08:29:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 08:29:17.725455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413284ac-89a7-4b70-99bc-c7d7b688dbdb None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:17.794043 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:17.834131 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413284ac-89a7-4b70-99bc-c7d7b688dbdb None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:17.868890 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa946f48-44e8-401c-a76a-97027bc66b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:17.869730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa946f48-44e8-401c-a76a-97027bc66b77 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:17.870013 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa946f48-44e8-401c-a76a-97027bc66b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:17.870291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa946f48-44e8-401c-a76a-97027bc66b77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:17.881353 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa946f48-44e8-401c-a76a-97027bc66b77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:17.883169 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa946f48-44e8-401c-a76a-97027bc66b77 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:17.901154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-fa946f48-44e8-401c-a76a-97027bc66b77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:29:17.901154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:17.901154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:17.901154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:17.901154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:17.901154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:17.901154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:17.901154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:17.901154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:17.901154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:17.901154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:17.901154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:17.901154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 08:29:17.901154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 08:29:17.901154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 08:29:17.902432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 08:29:17.902432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:17.902432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:17.902432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:17.902432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:17.902432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:17.902432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:17.902432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:17.902432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:17.902432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:17.902432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:17.902432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:17.902432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:17.902432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:17.902432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:17.902432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:17.902432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:17.903642 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 08:29:17.903642 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 08:29:17.903642 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 08:29:17.903642 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:29:17.903642 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 639/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:17.908519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09b651-42f3-4520-9c05-14fa6a9add62 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:18.014427 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09b651-42f3-4520-9c05-14fa6a9add62 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:18.050275 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87652ab5-b892-4173-8ec9-e4d86c4474b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:18.050948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87652ab5-b892-4173-8ec9-e4d86c4474b9 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:18.051243 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87652ab5-b892-4173-8ec9-e4d86c4474b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:18.051523 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87652ab5-b892-4173-8ec9-e4d86c4474b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:18.052352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87652ab5-b892-4173-8ec9-e4d86c4474b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:18.052665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87652ab5-b892-4173-8ec9-e4d86c4474b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:18.054049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87652ab5-b892-4173-8ec9-e4d86c4474b9 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:18.066640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 645/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Jul 18 08:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:29:18.072506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa946f48-44e8-401c-a76a-97027bc66b77 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:18.180640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa946f48-44e8-401c-a76a-97027bc66b77 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:18.195109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:18.216517 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48760278-0cee-4727-91b0-4921e773d398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:18.217929 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48760278-0cee-4727-91b0-4921e773d398 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:18.218288 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48760278-0cee-4727-91b0-4921e773d398 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:18.218595 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48760278-0cee-4727-91b0-4921e773d398 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:18.225799 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48760278-0cee-4727-91b0-4921e773d398 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:18.227249 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48760278-0cee-4727-91b0-4921e773d398 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:18.242894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application [None req-48760278-0cee-4727-91b0-4921e773d398 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:29:18.242894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:18.242894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:18.242894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:18.242894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:18.242894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:18.242894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:18.242894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:18.242894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:18.242894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:18.242894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:18.242894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:18.242894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 08:29:18.242894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 08:29:18.242894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 08:29:18.244025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 08:29:18.244025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:18.244025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:18.244025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:18.244025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:18.244025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:18.244025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:18.244025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:18.244025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:18.244025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:18.244025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:18.244025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:18.244025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:18.244025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:18.244025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:18.244025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:18.244025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:18.245107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 08:29:18.245107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 08:29:18.245107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 08:29:18.245107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 08:29:18.245107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 08:29:18.245107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 08:29:18.245107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 08:29:18.245107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: ERROR keystone.server.flask.application Jul 18 08:29:18.245107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 640/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Jul 18 08:29:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:18.251156 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87652ab5-b892-4173-8ec9-e4d86c4474b9 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:18.307322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:18.362209 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87652ab5-b892-4173-8ec9-e4d86c4474b9 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:18.395499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ef5bd-d966-47c4-88e4-5da3659003a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:18.396198 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ef5bd-d966-47c4-88e4-5da3659003a6 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:18.396484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ef5bd-d966-47c4-88e4-5da3659003a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:18.396759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ef5bd-d966-47c4-88e4-5da3659003a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:18.397546 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ef5bd-d966-47c4-88e4-5da3659003a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:18.397856 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ef5bd-d966-47c4-88e4-5da3659003a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:18.399299 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ef5bd-d966-47c4-88e4-5da3659003a6 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:18.412541 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 646/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Jul 18 08:29:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:18.422516 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48760278-0cee-4727-91b0-4921e773d398 None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:18.540206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48760278-0cee-4727-91b0-4921e773d398 None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:18.577628 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955018e-7fe7-4b46-8c33-27fb6f0d5e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:18.578183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955018e-7fe7-4b46-8c33-27fb6f0d5e52 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:18.578471 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955018e-7fe7-4b46-8c33-27fb6f0d5e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:18.578743 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955018e-7fe7-4b46-8c33-27fb6f0d5e52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:18.579568 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955018e-7fe7-4b46-8c33-27fb6f0d5e52 None admin] RBAC: Adding query filter params (role.id=467d49b8a0d3435aaba58464cc8ef85c, scope.project.id=35c846931fd443fbb83a6fbe0aeaa866, user.id=d606fd7ef269460a8110223932f02d19) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:18.579911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955018e-7fe7-4b46-8c33-27fb6f0d5e52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:18.581345 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955018e-7fe7-4b46-8c33-27fb6f0d5e52 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:18.588028 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 641/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Jul 18 08:29:18 2022] GET /identity/v3/role_assignments?user.id=d606fd7ef269460a8110223932f02d19&scope.project.id=35c846931fd443fbb83a6fbe0aeaa866&role.id=467d49b8a0d3435aaba58464cc8ef85c => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:20.653948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307dc2e-0804-4b95-a640-34e9c4c15936 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:20.655114 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307dc2e-0804-4b95-a640-34e9c4c15936 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:20.655114 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307dc2e-0804-4b95-a640-34e9c4c15936 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:20.656488 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 647/1288] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:29:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:29:20.661811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea98bf-b979-43f8-bba7-8d3cc8ef5256 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:20.662242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea98bf-b979-43f8-bba7-8d3cc8ef5256 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:20.662346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea98bf-b979-43f8-bba7-8d3cc8ef5256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:20.680365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-cdea98bf-b979-43f8-bba7-8d3cc8ef5256 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:29:20.681540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-cdea98bf-b979-43f8-bba7-8d3cc8ef5256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:29:20.712670 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:20.751577 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 642/1289] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:29:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:29:20.774506 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307dc2e-0804-4b95-a640-34e9c4c15936 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:20.882360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307dc2e-0804-4b95-a640-34e9c4c15936 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:20.894048 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:20.919754 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9efae-f95a-4671-af34-01e6b5ac0f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:20.920865 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9efae-f95a-4671-af34-01e6b5ac0f18 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:20.921186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9efae-f95a-4671-af34-01e6b5ac0f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:20.921463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9efae-f95a-4671-af34-01e6b5ac0f18 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:20.922113 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9efae-f95a-4671-af34-01e6b5ac0f18 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:20.923476 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9efae-f95a-4671-af34-01e6b5ac0f18 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:20.935057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-52b9efae-f95a-4671-af34-01e6b5ac0f18 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 08:29:20.935057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:20.935057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:29:20.935057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:29:20.935057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:29:20.935057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:29:20.935057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:29:20.935057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:29:20.935057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 08:29:20.935057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:29:20.935057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:29:20.935057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:29:20.935057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:20.935057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:20.935057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:20.935057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:20.935057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:20.936407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:20.936407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:20.936407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:20.936407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:20.936407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:20.936407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:20.936407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:29:20.936407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:29:20.936407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:29:20.936407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:29:20.936407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:20.936407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:20.936407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:20.936407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:20.936407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:20.936407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:20.937473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:20.937473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:20.937473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:20.937473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:20.937473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:20.937473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:20.937473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:20.937473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:20.937473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:20.937473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:20.937473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:29:20.937473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:29:20.937473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 08:29:20.937473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:29:20.937473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 648/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 18 08:29:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:20.942749 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea98bf-b979-43f8-bba7-8d3cc8ef5256 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:21.045242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea98bf-b979-43f8-bba7-8d3cc8ef5256 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:21.078635 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780da97-051e-4340-9387-65255609c602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:21.079297 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780da97-051e-4340-9387-65255609c602 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:21.079581 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780da97-051e-4340-9387-65255609c602 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:21.079887 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780da97-051e-4340-9387-65255609c602 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:21.080674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780da97-051e-4340-9387-65255609c602 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:21.080982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780da97-051e-4340-9387-65255609c602 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:21.082370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780da97-051e-4340-9387-65255609c602 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:21.087289 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 643/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Jul 18 08:29:20 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:21.094151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9efae-f95a-4671-af34-01e6b5ac0f18 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:21.197122 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9efae-f95a-4671-af34-01e6b5ac0f18 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:21.231456 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ad989-fecb-4b23-bdf4-7f38c29b98e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:21.232143 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ad989-fecb-4b23-bdf4-7f38c29b98e8 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:21.232452 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ad989-fecb-4b23-bdf4-7f38c29b98e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:21.232784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ad989-fecb-4b23-bdf4-7f38c29b98e8 None admin] PATH_INFO: `/v3/services` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:21.233816 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ad989-fecb-4b23-bdf4-7f38c29b98e8 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:21.234124 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ad989-fecb-4b23-bdf4-7f38c29b98e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:21.235488 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ad989-fecb-4b23-bdf4-7f38c29b98e8 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:21.240135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 649/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Jul 18 08:29:21 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:23.330915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca588c-57b0-4775-aaf8-1519a609a46e None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:23.331517 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca588c-57b0-4775-aaf8-1519a609a46e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:23.331760 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca588c-57b0-4775-aaf8-1519a609a46e None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:23.333405 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 644/1293] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:29:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:29:23.338691 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435b338-83a1-4d32-a978-53ca3e66af73 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:23.339154 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435b338-83a1-4d32-a978-53ca3e66af73 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:23.339250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435b338-83a1-4d32-a978-53ca3e66af73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:23.357240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-4435b338-83a1-4d32-a978-53ca3e66af73 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:29:23.358581 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4435b338-83a1-4d32-a978-53ca3e66af73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:29:23.389250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:23.433717 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 650/1294] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:29:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:29:23.457757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca588c-57b0-4775-aaf8-1519a609a46e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:23.490629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:23.567461 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca588c-57b0-4775-aaf8-1519a609a46e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:23.601883 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c390605-8b58-4f2c-bfff-01d3becac5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:23.602463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c390605-8b58-4f2c-bfff-01d3becac5fc None admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:23.602738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c390605-8b58-4f2c-bfff-01d3becac5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:23.603007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c390605-8b58-4f2c-bfff-01d3becac5fc None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:23.603742 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c390605-8b58-4f2c-bfff-01d3becac5fc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:23.605355 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c390605-8b58-4f2c-bfff-01d3becac5fc None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:23.718747 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 645/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Jul 18 08:29:23 2022] POST /identity/v3/services => generated 246 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:29:25.838535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1af016-0713-4143-af72-4a0ab90851e1 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:25.838535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1af016-0713-4143-af72-4a0ab90851e1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:25.838992 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1af016-0713-4143-af72-4a0ab90851e1 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:25.840136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 651/1296] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:29:25.845922 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aac08c-ebb8-4528-b015-e1429fc0c9f4 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:25.849285 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aac08c-ebb8-4528-b015-e1429fc0c9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:25.849285 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aac08c-ebb8-4528-b015-e1429fc0c9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:25.863988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a0aac08c-ebb8-4528-b015-e1429fc0c9f4 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:29:25.865240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a0aac08c-ebb8-4528-b015-e1429fc0c9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:29:25.940981 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 646/1297] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:29:25 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:29:25.964234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1af016-0713-4143-af72-4a0ab90851e1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:26.073212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1af016-0713-4143-af72-4a0ab90851e1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:26.107263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac7032-069b-4094-84d3-62c058b226e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:26.108885 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac7032-069b-4094-84d3-62c058b226e5 None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:26.109171 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac7032-069b-4094-84d3-62c058b226e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:26.109442 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac7032-069b-4094-84d3-62c058b226e5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:26.110091 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac7032-069b-4094-84d3-62c058b226e5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:26.111460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac7032-069b-4094-84d3-62c058b226e5 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:26.123367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-03ac7032-069b-4094-84d3-62c058b226e5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 08:29:26.123367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:26.123367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:29:26.123367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:29:26.123367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:29:26.123367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:29:26.123367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:29:26.123367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:29:26.123367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 08:29:26.123367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:29:26.123367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:29:26.123367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:29:26.123367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:26.123367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:26.123367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:26.123367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:26.123367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:26.124434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:26.124434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:26.124434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:26.124434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:26.124434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:26.124434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:26.124434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:29:26.124434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:29:26.124434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:29:26.124434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:29:26.124434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:26.124434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:26.124434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:26.124434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:26.124434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:26.124434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:26.125343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:26.125343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:26.125343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:26.125343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:26.125343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:26.125343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:26.125343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:26.125343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:26.125343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:26.125343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:26.125343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:29:26.125343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:29:26.125343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 08:29:26.125343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:29:26.125343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 652/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 18 08:29:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:26.130852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aac08c-ebb8-4528-b015-e1429fc0c9f4 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:26.149269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:26.227613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aac08c-ebb8-4528-b015-e1429fc0c9f4 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:26.261045 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecf5db-8ac7-4936-be41-621f8edd612b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:26.261704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecf5db-8ac7-4936-be41-621f8edd612b None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:26.261983 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecf5db-8ac7-4936-be41-621f8edd612b None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:26.262254 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecf5db-8ac7-4936-be41-621f8edd612b None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:26.262956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ecf5db-8ac7-4936-be41-621f8edd612b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:26.263275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ecf5db-8ac7-4936-be41-621f8edd612b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:26.265691 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ecf5db-8ac7-4936-be41-621f8edd612b None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:26.271472 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 647/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Jul 18 08:29:26 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:26.277756 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac7032-069b-4094-84d3-62c058b226e5 None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:26.309639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:26.378539 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac7032-069b-4094-84d3-62c058b226e5 None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:26.412949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f91321-34cd-4a4f-9358-964806ca4a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:26.413603 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f91321-34cd-4a4f-9358-964806ca4a2c None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:26.413938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f91321-34cd-4a4f-9358-964806ca4a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:26.414218 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f91321-34cd-4a4f-9358-964806ca4a2c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:26.415001 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f91321-34cd-4a4f-9358-964806ca4a2c None admin] RBAC: Adding query filter params (interface=public, service_id=13a4719034c540cea7e737d4f402bb73, region_id=RegionOne) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:26.415311 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f91321-34cd-4a4f-9358-964806ca4a2c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:26.416698 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f91321-34cd-4a4f-9358-964806ca4a2c None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:26.422347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 653/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Jul 18 08:29:26 2022] GET /identity/v3/endpoints?service_id=13a4719034c540cea7e737d4f402bb73&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:29:26.428161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecf5db-8ac7-4936-be41-621f8edd612b None admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:26.513331 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:26.532906 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecf5db-8ac7-4936-be41-621f8edd612b None admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:26.567841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92ad6f-e998-4ee0-b54b-fe7b10a3f696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:26.568512 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92ad6f-e998-4ee0-b54b-fe7b10a3f696 None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:26.568799 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92ad6f-e998-4ee0-b54b-fe7b10a3f696 None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:26.569076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92ad6f-e998-4ee0-b54b-fe7b10a3f696 None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:26.569789 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d92ad6f-e998-4ee0-b54b-fe7b10a3f696 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:26.571179 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d92ad6f-e998-4ee0-b54b-fe7b10a3f696 None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:26.578387 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 648/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Jul 18 08:29:26 2022] GET /identity/v3/services => generated 2191 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:29:28.627719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1812d5e-14e9-4109-a54a-04588e9fa818 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:28.627719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1812d5e-14e9-4109-a54a-04588e9fa818 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:28.628332 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1812d5e-14e9-4109-a54a-04588e9fa818 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:28.629465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 654/1302] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:29:28.634907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ebfc9-0f2a-409f-97b6-b798292552ba None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:28.635378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ebfc9-0f2a-409f-97b6-b798292552ba None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:28.635474 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ebfc9-0f2a-409f-97b6-b798292552ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:28.653777 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-344ebfc9-0f2a-409f-97b6-b798292552ba None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:29:28.655564 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-344ebfc9-0f2a-409f-97b6-b798292552ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:29:28.726375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 649/1303] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:29:28 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 08:29:28.752216 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1812d5e-14e9-4109-a54a-04588e9fa818 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:28.855591 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:28.862578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1812d5e-14e9-4109-a54a-04588e9fa818 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:28.899034 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54307a4f-a48f-48ba-89eb-9e7c9514a3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:28.899786 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54307a4f-a48f-48ba-89eb-9e7c9514a3df None admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:28.900080 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54307a4f-a48f-48ba-89eb-9e7c9514a3df None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:28.900458 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54307a4f-a48f-48ba-89eb-9e7c9514a3df None admin] PATH_INFO: `/v3/services/placement` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:28.901116 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54307a4f-a48f-48ba-89eb-9e7c9514a3df None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:28.902492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54307a4f-a48f-48ba-89eb-9e7c9514a3df None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:28.914646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application [None req-54307a4f-a48f-48ba-89eb-9e7c9514a3df None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 08:29:28.914646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:28.914646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 08:29:28.914646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 08:29:28.914646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 08:29:28.914646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 08:29:28.914646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 08:29:28.914646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:29:28.914646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 08:29:28.914646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:29:28.914646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 08:29:28.914646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:29:28.914646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 08:29:28.914646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 08:29:28.914646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 08:29:28.914646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 08:29:28.914646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 08:29:28.915876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 08:29:28.915876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 08:29:28.915876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 08:29:28.915876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 08:29:28.915876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 08:29:28.915876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 08:29:28.915876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 08:29:28.915876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 08:29:28.915876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 08:29:28.915876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 08:29:28.915876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 08:29:28.915876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 08:29:28.915876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 08:29:28.915876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 08:29:28.915876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 08:29:28.915876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 08:29:28.916752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 08:29:28.916752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application with Lock( Jul 18 08:29:28.916752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 08:29:28.916752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self._enter() Jul 18 08:29:28.916752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 08:29:28.916752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 08:29:28.916752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 08:29:28.916752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application return self.creator() Jul 18 08:29:28.916752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 08:29:28.916752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application created_value = creator( Jul 18 08:29:28.916752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 08:29:28.916752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 08:29:28.916752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 08:29:28.916752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: ERROR keystone.server.flask.application Jul 18 08:29:28.916752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 655/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Jul 18 08:29:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 08:29:28.922676 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ebfc9-0f2a-409f-97b6-b798292552ba None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:29.000100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:29.023466 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ebfc9-0f2a-409f-97b6-b798292552ba None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:29.057911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0380d-41e3-42c5-8c19-a462c000029a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:29.058580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0380d-41e3-42c5-8c19-a462c000029a None admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:29.058860 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0380d-41e3-42c5-8c19-a462c000029a None admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:29.059134 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0380d-41e3-42c5-8c19-a462c000029a None admin] PATH_INFO: `/v3/services` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:29.059880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0380d-41e3-42c5-8c19-a462c000029a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:29:29.060176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0380d-41e3-42c5-8c19-a462c000029a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:29.062231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0380d-41e3-42c5-8c19-a462c000029a None admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:29.068455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 650/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Jul 18 08:29:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:29:29.073478 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54307a4f-a48f-48ba-89eb-9e7c9514a3df None admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:29:29.176549 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54307a4f-a48f-48ba-89eb-9e7c9514a3df None admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:29:29.210903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b1518-4034-40b4-be00-0b4d836ce535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:29:29.211565 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b1518-4034-40b4-be00-0b4d836ce535 None admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:29:29.211875 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b1518-4034-40b4-be00-0b4d836ce535 None admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:29:29.212150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b1518-4034-40b4-be00-0b4d836ce535 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:29:29.212902 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b1518-4034-40b4-be00-0b4d836ce535 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:29:29.214427 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b1518-4034-40b4-be00-0b4d836ce535 None admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:29:29.294160 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 656/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Jul 18 08:29:29 2022] POST /identity/v3/endpoints => generated 331 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:29:39.064429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=30 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:29:39.289268 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:36:54.247289 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff77a50-9609-4bc0-960c-7b9d80d4ab22 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:36:54.247793 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff77a50-9609-4bc0-960c-7b9d80d4ab22 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:36:54.248088 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff77a50-9609-4bc0-960c-7b9d80d4ab22 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:36:54.250477 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 651/1307] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:36:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:36:54.256397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff851fdf-d9c9-43a9-bb08-05933b448dc3 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:36:54.256851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff851fdf-d9c9-43a9-bb08-05933b448dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:36:54.256980 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff851fdf-d9c9-43a9-bb08-05933b448dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:36:54.392813 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ff851fdf-d9c9-43a9-bb08-05933b448dc3 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:36:54.395239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ff851fdf-d9c9-43a9-bb08-05933b448dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:36:54.529201 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 657/1308] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:36:54.581539 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e238676-dfda-4c29-8444-5c02a92f9ff3 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:36:54.581889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e238676-dfda-4c29-8444-5c02a92f9ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:36:54.582168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e238676-dfda-4c29-8444-5c02a92f9ff3 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:36:54.583549 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 652/1309] 10.222.0.57 () {54 vars in 965 bytes} [Mon Jul 18 08:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:36:54.589816 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f25e2b-3330-47a7-9215-7915002ce3a9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:36:54.590261 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f25e2b-3330-47a7-9215-7915002ce3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:36:54.590642 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f25e2b-3330-47a7-9215-7915002ce3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:36:54.675085 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-43f25e2b-3330-47a7-9215-7915002ce3a9 None None] MFA Rules not processed for user `24a6b794f5774e35b67e9ec4358ffc66`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:36:54.676944 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-43f25e2b-3330-47a7-9215-7915002ce3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:36:54.705198 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:36:54.771273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 658/1310] 10.222.0.57 () {58 vars in 1062 bytes} [Mon Jul 18 08:36:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:36:54.780624 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e238676-dfda-4c29-8444-5c02a92f9ff3 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:36:54.782542 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4e238676-dfda-4c29-8444-5c02a92f9ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:36:54.916008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e238676-dfda-4c29-8444-5c02a92f9ff3 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:36:55.002227 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c3913-eb53-4dc4-96fe-871719622a30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:36:55.002916 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c3913-eb53-4dc4-96fe-871719622a30 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:36:55.003196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c3913-eb53-4dc4-96fe-871719622a30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:36:55.003471 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c3913-eb53-4dc4-96fe-871719622a30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:36:55.005707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-120c3913-eb53-4dc4-96fe-871719622a30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:36:55.072335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:36:55.162095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c3913-eb53-4dc4-96fe-871719622a30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:36:55.163565 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c3913-eb53-4dc4-96fe-871719622a30 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:36:55.189390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 653/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:36:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:36:57.587591 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41cd8d-9a87-4143-8ef6-9ef9b607a3d0 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:36:57.588317 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41cd8d-9a87-4143-8ef6-9ef9b607a3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:36:57.588317 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41cd8d-9a87-4143-8ef6-9ef9b607a3d0 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:36:57.589772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 659/1312] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:36:57.596284 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec9318-58ca-4022-8c3b-4d2165085df1 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:36:57.596866 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec9318-58ca-4022-8c3b-4d2165085df1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:36:57.596866 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec9318-58ca-4022-8c3b-4d2165085df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:36:57.714327 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-49ec9318-58ca-4022-8c3b-4d2165085df1 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:36:57.715717 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-49ec9318-58ca-4022-8c3b-4d2165085df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:36:57.885690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 654/1313] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:36:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:36:57.963833 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cd9a9-627a-4cbc-bb3d-504969a30b67 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:36:57.964123 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cd9a9-627a-4cbc-bb3d-504969a30b67 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:36:57.964402 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cd9a9-627a-4cbc-bb3d-504969a30b67 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:36:57.965767 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 660/1314] 10.222.0.57 () {54 vars in 965 bytes} [Mon Jul 18 08:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:36:57.972388 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76bd30-8dc8-410f-a13f-04623e64b9b7 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:36:57.972745 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76bd30-8dc8-410f-a13f-04623e64b9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:36:57.973461 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76bd30-8dc8-410f-a13f-04623e64b9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:36:57.986559 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:36:58.065976 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-4a76bd30-8dc8-410f-a13f-04623e64b9b7 None None] MFA Rules not processed for user `24a6b794f5774e35b67e9ec4358ffc66`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:36:58.067885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4a76bd30-8dc8-410f-a13f-04623e64b9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:36:58.150340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 655/1315] 10.222.0.57 () {58 vars in 1062 bytes} [Mon Jul 18 08:36:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:36:58.160451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537cd9a9-627a-4cbc-bb3d-504969a30b67 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:36:58.162113 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-537cd9a9-627a-4cbc-bb3d-504969a30b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:36:58.272942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537cd9a9-627a-4cbc-bb3d-504969a30b67 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:36:58.350069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:36:58.353257 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7fc6c-d841-49ed-abf7-1daae3d4678b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:36:58.353990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7fc6c-d841-49ed-abf7-1daae3d4678b service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:36:58.354371 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7fc6c-d841-49ed-abf7-1daae3d4678b service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:36:58.354838 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7fc6c-d841-49ed-abf7-1daae3d4678b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:36:58.357361 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-54f7fc6c-d841-49ed-abf7-1daae3d4678b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:36:58.485147 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7fc6c-d841-49ed-abf7-1daae3d4678b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:36:58.486609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7fc6c-d841-49ed-abf7-1daae3d4678b service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:36:58.507069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 661/1316] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:36:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:37:01.545748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7da55-6321-456f-b2df-97626c3735ad None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:01.546583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7da55-6321-456f-b2df-97626c3735ad None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:01.546583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7da55-6321-456f-b2df-97626c3735ad None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:01.547750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 656/1317] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:37:01.553502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5274880-a9c2-40b7-a925-336eaa798a93 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:01.553983 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5274880-a9c2-40b7-a925-336eaa798a93 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:01.554076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5274880-a9c2-40b7-a925-336eaa798a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:01.626585 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c5274880-a9c2-40b7-a925-336eaa798a93 None None] MFA Rules not processed for user `a9e3e8eb9c8c4db0ae0629919a5b1a56`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:37:01.627845 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c5274880-a9c2-40b7-a925-336eaa798a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:01.681545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:01.729423 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 662/1318] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:37:01 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:37:01.770300 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7da55-6321-456f-b2df-97626c3735ad None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:37:01.771950 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-08f7da55-6321-456f-b2df-97626c3735ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:01.882401 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7da55-6321-456f-b2df-97626c3735ad None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:37:01.885308 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:01.960433 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3b5bb-aca1-4667-93fa-6e1b5911af69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:37:01.961095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3b5bb-aca1-4667-93fa-6e1b5911af69 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:01.961377 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3b5bb-aca1-4667-93fa-6e1b5911af69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:01.961654 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3b5bb-aca1-4667-93fa-6e1b5911af69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:01.963342 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d8b3b5bb-aca1-4667-93fa-6e1b5911af69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:02.087001 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3b5bb-aca1-4667-93fa-6e1b5911af69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:37:02.088689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3b5bb-aca1-4667-93fa-6e1b5911af69 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:37:02.108702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 657/1319] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:37:01 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:37:05.196894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b382ee-93bc-41a7-9391-1df2cfea1b4e None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:05.197524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b382ee-93bc-41a7-9391-1df2cfea1b4e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:05.197524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b382ee-93bc-41a7-9391-1df2cfea1b4e None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:05.198850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 663/1320] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:37:05.204901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f8559-aa6d-4cbc-98eb-115f08efdad6 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:05.205394 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f8559-aa6d-4cbc-98eb-115f08efdad6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:05.205556 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f8559-aa6d-4cbc-98eb-115f08efdad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:05.247973 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:05.274397 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-880f8559-aa6d-4cbc-98eb-115f08efdad6 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:37:05.275599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-880f8559-aa6d-4cbc-98eb-115f08efdad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:05.380414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 658/1321] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:37:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:37:05.421509 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b382ee-93bc-41a7-9391-1df2cfea1b4e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:37:05.423084 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-87b382ee-93bc-41a7-9391-1df2cfea1b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:05.534178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b382ee-93bc-41a7-9391-1df2cfea1b4e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:37:05.612306 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ddc02-cf1b-4b1a-9d48-221e7965f470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:37:05.612955 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ddc02-cf1b-4b1a-9d48-221e7965f470 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:05.613238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ddc02-cf1b-4b1a-9d48-221e7965f470 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:05.613512 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ddc02-cf1b-4b1a-9d48-221e7965f470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:05.615180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6a8ddc02-cf1b-4b1a-9d48-221e7965f470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:05.622442 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:05.733989 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8ddc02-cf1b-4b1a-9d48-221e7965f470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:37:05.735481 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8ddc02-cf1b-4b1a-9d48-221e7965f470 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:37:05.756636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 664/1322] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:37:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:37:08.969504 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59392ac4-d853-479a-bef1-7b6efccf9596 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:08.970066 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59392ac4-d853-479a-bef1-7b6efccf9596 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:08.970066 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59392ac4-d853-479a-bef1-7b6efccf9596 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:08.971481 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 659/1323] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:37:08.977583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa62c4-fb6f-4540-ad3e-273fc09de41f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:08.978413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa62c4-fb6f-4540-ad3e-273fc09de41f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:08.978413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa62c4-fb6f-4540-ad3e-273fc09de41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:09.054948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-9cfa62c4-fb6f-4540-ad3e-273fc09de41f None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:37:09.056245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9cfa62c4-fb6f-4540-ad3e-273fc09de41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:09.121676 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:09.169607 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 665/1324] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:37:09.208110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59392ac4-d853-479a-bef1-7b6efccf9596 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:37:09.209654 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-59392ac4-d853-479a-bef1-7b6efccf9596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:09.345729 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59392ac4-d853-479a-bef1-7b6efccf9596 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:37:09.360437 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:09.432895 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c2228-5afe-4cbe-89ee-fd60aa24538c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:37:09.433467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c2228-5afe-4cbe-89ee-fd60aa24538c service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:09.433751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c2228-5afe-4cbe-89ee-fd60aa24538c service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:09.434025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c2228-5afe-4cbe-89ee-fd60aa24538c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:09.436075 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-592c2228-5afe-4cbe-89ee-fd60aa24538c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:09.565223 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c2228-5afe-4cbe-89ee-fd60aa24538c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:37:09.566658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c2228-5afe-4cbe-89ee-fd60aa24538c service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:37:09.587091 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 660/1325] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:37:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:37:11.716441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c1ff5-1486-4dc4-a3fb-37fdff78fb1a None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:11.717241 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c1ff5-1486-4dc4-a3fb-37fdff78fb1a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:11.717241 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c1ff5-1486-4dc4-a3fb-37fdff78fb1a None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:11.718323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 666/1326] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:37:11.723994 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b147b24-07aa-402a-b71b-f2330634aa37 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:11.724447 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b147b24-07aa-402a-b71b-f2330634aa37 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:11.724551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b147b24-07aa-402a-b71b-f2330634aa37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:11.774594 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:11.794150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-4b147b24-07aa-402a-b71b-f2330634aa37 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:37:11.795762 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4b147b24-07aa-402a-b71b-f2330634aa37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:11.906130 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 661/1327] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:37:11.942523 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c1ff5-1486-4dc4-a3fb-37fdff78fb1a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:37:11.944181 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c10c1ff5-1486-4dc4-a3fb-37fdff78fb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:12.053934 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c1ff5-1486-4dc4-a3fb-37fdff78fb1a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:37:12.129020 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb1567-89b3-4829-b49d-a77959c90537 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:37:12.129587 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb1567-89b3-4829-b49d-a77959c90537 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:12.129873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb1567-89b3-4829-b49d-a77959c90537 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:12.130153 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb1567-89b3-4829-b49d-a77959c90537 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:12.131856 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-39cb1567-89b3-4829-b49d-a77959c90537 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:12.149778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:12.250123 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb1567-89b3-4829-b49d-a77959c90537 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:37:12.251620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb1567-89b3-4829-b49d-a77959c90537 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:37:12.273041 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 667/1328] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:37:14.359279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9544e2a-f7f5-4c66-8563-8103c5c92f09 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:14.359908 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9544e2a-f7f5-4c66-8563-8103c5c92f09 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:14.359908 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9544e2a-f7f5-4c66-8563-8103c5c92f09 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:14.361242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 662/1329] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:37:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:37:14.366602 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84935854-ac79-49fd-aaa8-68d1c7c50ac0 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:14.367068 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84935854-ac79-49fd-aaa8-68d1c7c50ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:14.367172 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84935854-ac79-49fd-aaa8-68d1c7c50ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:14.444351 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-84935854-ac79-49fd-aaa8-68d1c7c50ac0 None None] MFA Rules not processed for user `a9e3e8eb9c8c4db0ae0629919a5b1a56`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:37:14.445559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-84935854-ac79-49fd-aaa8-68d1c7c50ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:14.510891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:14.546945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 668/1330] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:37:14 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:37:14.585437 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9544e2a-f7f5-4c66-8563-8103c5c92f09 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:37:14.586880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f9544e2a-f7f5-4c66-8563-8103c5c92f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:14.696472 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9544e2a-f7f5-4c66-8563-8103c5c92f09 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:37:14.719851 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:14.775110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34198be-8d29-4471-92ab-0fc6daae0dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:37:14.775795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34198be-8d29-4471-92ab-0fc6daae0dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:14.776074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34198be-8d29-4471-92ab-0fc6daae0dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:14.776340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34198be-8d29-4471-92ab-0fc6daae0dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:14.777969 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d34198be-8d29-4471-92ab-0fc6daae0dd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:14.893685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34198be-8d29-4471-92ab-0fc6daae0dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:37:14.895123 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34198be-8d29-4471-92ab-0fc6daae0dd3 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:37:14.914226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 663/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:37:14 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:37:17.577416 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c795af-9624-4ffc-8335-89da258b8d63 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:17.577967 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c795af-9624-4ffc-8335-89da258b8d63 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:17.577967 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c795af-9624-4ffc-8335-89da258b8d63 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:17.580220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 669/1332] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:37:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:37:17.586146 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9958db-7524-4248-94ba-20c49de6e351 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:17.586676 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9958db-7524-4248-94ba-20c49de6e351 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:17.586676 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9958db-7524-4248-94ba-20c49de6e351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:17.656563 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:17.659422 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-3c9958db-7524-4248-94ba-20c49de6e351 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:37:17.660733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3c9958db-7524-4248-94ba-20c49de6e351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:17.766343 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 664/1333] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:37:17.804936 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c795af-9624-4ffc-8335-89da258b8d63 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:37:17.806782 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c7c795af-9624-4ffc-8335-89da258b8d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:17.918080 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c795af-9624-4ffc-8335-89da258b8d63 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:37:18.001791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf00556-6190-4085-acb2-4fded252dc94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:37:18.002445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf00556-6190-4085-acb2-4fded252dc94 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:18.002728 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf00556-6190-4085-acb2-4fded252dc94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:18.003000 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf00556-6190-4085-acb2-4fded252dc94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:18.004744 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bdf00556-6190-4085-acb2-4fded252dc94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:18.037270 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:18.133175 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf00556-6190-4085-acb2-4fded252dc94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:37:18.134645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf00556-6190-4085-acb2-4fded252dc94 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:37:18.158884 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 670/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:37:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:37:20.261532 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7ed6c-1e4c-4850-869b-f2f037dc2993 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:20.262069 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7ed6c-1e4c-4850-869b-f2f037dc2993 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:20.262185 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7ed6c-1e4c-4850-869b-f2f037dc2993 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:20.264190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 665/1335] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:37:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:37:20.269616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7dd93-3550-4e1c-bb10-7b85417c76ad None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:20.270066 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7dd93-3550-4e1c-bb10-7b85417c76ad None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:20.270185 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7dd93-3550-4e1c-bb10-7b85417c76ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:20.337605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f2f7dd93-3550-4e1c-bb10-7b85417c76ad None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:37:20.338842 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f2f7dd93-3550-4e1c-bb10-7b85417c76ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:20.425322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:20.447947 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 671/1336] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:37:20.492808 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7ed6c-1e4c-4850-869b-f2f037dc2993 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:37:20.494404 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1bc7ed6c-1e4c-4850-869b-f2f037dc2993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:20.602943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7ed6c-1e4c-4850-869b-f2f037dc2993 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:37:20.631253 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:20.678753 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1623fc0-10a0-43ec-b18e-9d130cf89761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:37:20.679388 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1623fc0-10a0-43ec-b18e-9d130cf89761 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:20.679683 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1623fc0-10a0-43ec-b18e-9d130cf89761 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:20.679969 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1623fc0-10a0-43ec-b18e-9d130cf89761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:20.681611 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a1623fc0-10a0-43ec-b18e-9d130cf89761 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:20.791543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1623fc0-10a0-43ec-b18e-9d130cf89761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:37:20.793015 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1623fc0-10a0-43ec-b18e-9d130cf89761 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:37:20.812591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 666/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:37:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:37:23.515889 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e2e36-e229-451b-8eff-b4329188abd5 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:23.516418 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e2e36-e229-451b-8eff-b4329188abd5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:23.516554 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e2e36-e229-451b-8eff-b4329188abd5 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:23.517856 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 672/1338] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:37:23.523573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c908af0-347a-4599-84e4-52c4f6a125d2 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:23.524046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c908af0-347a-4599-84e4-52c4f6a125d2 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:23.524236 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c908af0-347a-4599-84e4-52c4f6a125d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:23.598988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0c908af0-347a-4599-84e4-52c4f6a125d2 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:37:23.600242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0c908af0-347a-4599-84e4-52c4f6a125d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:23.609482 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:23.705094 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 667/1339] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:37:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:37:23.743081 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e2e36-e229-451b-8eff-b4329188abd5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:37:23.744739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c84e2e36-e229-451b-8eff-b4329188abd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:23.859414 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e2e36-e229-451b-8eff-b4329188abd5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:37:23.938434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3fcfb-2f14-4f63-8c06-bc83961a4872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:37:23.939093 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3fcfb-2f14-4f63-8c06-bc83961a4872 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:23.939380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3fcfb-2f14-4f63-8c06-bc83961a4872 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:23.939689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3fcfb-2f14-4f63-8c06-bc83961a4872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:23.941359 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-69c3fcfb-2f14-4f63-8c06-bc83961a4872 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:23.977929 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:24.054604 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3fcfb-2f14-4f63-8c06-bc83961a4872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:37:24.056097 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3fcfb-2f14-4f63-8c06-bc83961a4872 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:37:24.074811 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 673/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:37:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:37:28.928736 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe5ac8-a29a-43e6-85aa-f1b3887f9c27 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:28.929323 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe5ac8-a29a-43e6-85aa-f1b3887f9c27 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:28.929323 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe5ac8-a29a-43e6-85aa-f1b3887f9c27 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:28.930718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 668/1341] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:37:28.936411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d6ebd-d708-47a8-a634-ebb8f52d2a19 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:28.936907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d6ebd-d708-47a8-a634-ebb8f52d2a19 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:28.937007 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d6ebd-d708-47a8-a634-ebb8f52d2a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:29.014219 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a69d6ebd-d708-47a8-a634-ebb8f52d2a19 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:37:29.015659 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a69d6ebd-d708-47a8-a634-ebb8f52d2a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:29.124641 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:29.138289 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 674/1342] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:37:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:37:29.181277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe5ac8-a29a-43e6-85aa-f1b3887f9c27 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:37:29.182909 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ffbe5ac8-a29a-43e6-85aa-f1b3887f9c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:29.290910 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe5ac8-a29a-43e6-85aa-f1b3887f9c27 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:37:29.330127 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:29.371839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a600f3d-0a3a-44d8-a242-b79f963fef11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:37:29.372498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a600f3d-0a3a-44d8-a242-b79f963fef11 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:29.372779 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a600f3d-0a3a-44d8-a242-b79f963fef11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:29.373054 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a600f3d-0a3a-44d8-a242-b79f963fef11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:29.374704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0a600f3d-0a3a-44d8-a242-b79f963fef11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:29.487484 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a600f3d-0a3a-44d8-a242-b79f963fef11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:37:29.489599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a600f3d-0a3a-44d8-a242-b79f963fef11 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:37:29.509871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 669/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:37:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:37:32.661469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b20f9-1f27-4eb2-a815-3b629786ef48 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:32.662036 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b20f9-1f27-4eb2-a815-3b629786ef48 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:32.662036 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b20f9-1f27-4eb2-a815-3b629786ef48 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:32.663535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 675/1344] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:37:32.669452 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360f566-baa4-4ec2-bcf5-eee0eae27839 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:32.670095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360f566-baa4-4ec2-bcf5-eee0eae27839 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:32.670095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360f566-baa4-4ec2-bcf5-eee0eae27839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:32.735805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0360f566-baa4-4ec2-bcf5-eee0eae27839 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:37:32.737002 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0360f566-baa4-4ec2-bcf5-eee0eae27839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:32.755609 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:32.840089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 670/1345] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:37:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:37:32.878621 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b20f9-1f27-4eb2-a815-3b629786ef48 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:37:32.880271 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-391b20f9-1f27-4eb2-a815-3b629786ef48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:32.997540 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b20f9-1f27-4eb2-a815-3b629786ef48 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:37:33.074996 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4af91-dff2-4cf3-a75b-ff291f3b95dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:37:33.075743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4af91-dff2-4cf3-a75b-ff291f3b95dc service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:33.076033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4af91-dff2-4cf3-a75b-ff291f3b95dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:33.076312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4af91-dff2-4cf3-a75b-ff291f3b95dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:33.078018 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-42c4af91-dff2-4cf3-a75b-ff291f3b95dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:33.124500 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:33.192434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4af91-dff2-4cf3-a75b-ff291f3b95dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:37:33.193889 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4af91-dff2-4cf3-a75b-ff291f3b95dc service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:37:33.214158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 676/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:37:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:37:37.546278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3242c-6c7a-4f4a-afa2-c3d4abcc0bbe None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:37.547139 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3242c-6c7a-4f4a-afa2-c3d4abcc0bbe None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:37.547323 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3242c-6c7a-4f4a-afa2-c3d4abcc0bbe None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:37.548902 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 671/1347] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:37:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:37:37.557474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44d6be-dd46-42b8-835b-7750fa91feae None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:37.557938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44d6be-dd46-42b8-835b-7750fa91feae None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:37.558330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44d6be-dd46-42b8-835b-7750fa91feae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:37.665585 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1a44d6be-dd46-42b8-835b-7750fa91feae None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:37:37.666841 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1a44d6be-dd46-42b8-835b-7750fa91feae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:37.781410 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:37.784306 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 677/1348] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:37:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:37:37.825463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3242c-6c7a-4f4a-afa2-c3d4abcc0bbe None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:37:37.827013 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-93a3242c-6c7a-4f4a-afa2-c3d4abcc0bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:37.949134 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3242c-6c7a-4f4a-afa2-c3d4abcc0bbe None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:37:38.006151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:38.040583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e069cb14-3303-4d5a-918c-2d8acbf7cb4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:37:38.041180 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069cb14-3303-4d5a-918c-2d8acbf7cb4c service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:38.041461 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069cb14-3303-4d5a-918c-2d8acbf7cb4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:38.041735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069cb14-3303-4d5a-918c-2d8acbf7cb4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:38.044463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e069cb14-3303-4d5a-918c-2d8acbf7cb4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:38.195217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e069cb14-3303-4d5a-918c-2d8acbf7cb4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:37:38.196800 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e069cb14-3303-4d5a-918c-2d8acbf7cb4c service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:37:38.218593 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 672/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:37:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:37:43.443457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a915a1a-9d6e-4f64-b0ef-6548a8085b1e None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:43.443940 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a915a1a-9d6e-4f64-b0ef-6548a8085b1e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:43.444047 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a915a1a-9d6e-4f64-b0ef-6548a8085b1e None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:43.445689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 678/1350] 10.222.0.57 () {54 vars in 921 bytes} [Mon Jul 18 08:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:37:43.457981 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f214e-1ca3-484c-ace6-6df1a4cf730c None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:43.458400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f214e-1ca3-484c-ace6-6df1a4cf730c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:43.461665 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f214e-1ca3-484c-ace6-6df1a4cf730c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:43.607435 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-ea0f214e-1ca3-484c-ace6-6df1a4cf730c None None] MFA Rules not processed for user `d606fd7ef269460a8110223932f02d19`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:37:43.609979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ea0f214e-1ca3-484c-ace6-6df1a4cf730c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:43.647343 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:43.828898 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 673/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Jul 18 08:37:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:37:48.280678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc905053-382a-4aa2-a765-a434bf7c23c3 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:48.281759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc905053-382a-4aa2-a765-a434bf7c23c3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:48.282172 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc905053-382a-4aa2-a765-a434bf7c23c3 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:48.283918 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 679/1352] 10.222.0.57 () {54 vars in 921 bytes} [Mon Jul 18 08:37:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:37:48.290990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb798633-3c5b-497e-b7cf-07488e928230 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:48.291965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb798633-3c5b-497e-b7cf-07488e928230 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:48.292937 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb798633-3c5b-497e-b7cf-07488e928230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:48.379481 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-eb798633-3c5b-497e-b7cf-07488e928230 None None] MFA Rules not processed for user `d606fd7ef269460a8110223932f02d19`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:37:48.381699 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-eb798633-3c5b-497e-b7cf-07488e928230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:48.460759 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:48.505633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 674/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Jul 18 08:37:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:37:49.873547 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3deb9c-295c-453b-963c-cc4518d4eda9 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:49.874073 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3deb9c-295c-453b-963c-cc4518d4eda9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:49.874198 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3deb9c-295c-453b-963c-cc4518d4eda9 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:49.875781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 680/1354] 10.222.0.57 () {54 vars in 921 bytes} [Mon Jul 18 08:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:37:49.881370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced39d5-193c-493b-81ff-59c6fc0217d3 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:49.882062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced39d5-193c-493b-81ff-59c6fc0217d3 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:49.882062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced39d5-193c-493b-81ff-59c6fc0217d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:49.975911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-9ced39d5-193c-493b-81ff-59c6fc0217d3 None None] MFA Rules not processed for user `d606fd7ef269460a8110223932f02d19`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:37:49.977172 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9ced39d5-193c-493b-81ff-59c6fc0217d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:50.118990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 675/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Jul 18 08:37:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:37:53.674513 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876bce7-d92a-43a3-aee0-a46fd0e7ffa9 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:53.675120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876bce7-d92a-43a3-aee0-a46fd0e7ffa9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:53.675120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876bce7-d92a-43a3-aee0-a46fd0e7ffa9 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:53.676503 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 681/1356] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:37:53.682719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502d3f1-28ea-4477-8c5c-cf5de63160b5 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:53.683919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502d3f1-28ea-4477-8c5c-cf5de63160b5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:53.683919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502d3f1-28ea-4477-8c5c-cf5de63160b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:53.745060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:53.803374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-e502d3f1-28ea-4477-8c5c-cf5de63160b5 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:37:53.806657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e502d3f1-28ea-4477-8c5c-cf5de63160b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:54.054714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 676/1357] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:37:54.132182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8fde3-0408-48a8-8ea4-d8d1d722106c None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:54.133234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8fde3-0408-48a8-8ea4-d8d1d722106c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:54.134048 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8fde3-0408-48a8-8ea4-d8d1d722106c None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:54.137125 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 682/1358] 10.222.0.57 () {54 vars in 962 bytes} [Mon Jul 18 08:37:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:37:54.143147 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b0d44-ef14-413d-b3cf-46c568263611 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:54.143716 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b0d44-ef14-413d-b3cf-46c568263611 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:54.144163 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b0d44-ef14-413d-b3cf-46c568263611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:54.294985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-8b6b0d44-ef14-413d-b3cf-46c568263611 None None] MFA Rules not processed for user `84cba474fc7c470ea119de697b9d7ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:37:54.298273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8b6b0d44-ef14-413d-b3cf-46c568263611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:54.380660 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:54.497430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 677/1359] 10.222.0.57 () {58 vars in 1059 bytes} [Mon Jul 18 08:37:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:37:54.509679 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8fde3-0408-48a8-8ea4-d8d1d722106c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:37:54.512253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-72f8fde3-0408-48a8-8ea4-d8d1d722106c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:54.684037 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8fde3-0408-48a8-8ea4-d8d1d722106c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:37:54.782520 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:54.831804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6167cd4-cc14-4542-afe4-4d9900de8fa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:37:54.832459 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6167cd4-cc14-4542-afe4-4d9900de8fa1 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:54.832775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6167cd4-cc14-4542-afe4-4d9900de8fa1 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:54.835917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6167cd4-cc14-4542-afe4-4d9900de8fa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:54.835917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f6167cd4-cc14-4542-afe4-4d9900de8fa1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:54.937866 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6167cd4-cc14-4542-afe4-4d9900de8fa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:37:54.939498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6167cd4-cc14-4542-afe4-4d9900de8fa1 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:37:54.972744 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 683/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:37:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:37:57.854055 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836799b-513b-4da0-ba34-9b70564ec5f5 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:57.855064 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836799b-513b-4da0-ba34-9b70564ec5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:57.855547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836799b-513b-4da0-ba34-9b70564ec5f5 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:57.857888 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 678/1361] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:37:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:37:57.867141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24959b7b-8b93-4e73-8f9b-1578c8f5d842 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:57.867873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24959b7b-8b93-4e73-8f9b-1578c8f5d842 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:57.868156 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24959b7b-8b93-4e73-8f9b-1578c8f5d842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:57.997400 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-24959b7b-8b93-4e73-8f9b-1578c8f5d842 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:37:57.998625 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-24959b7b-8b93-4e73-8f9b-1578c8f5d842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:58.045719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:58.134404 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 684/1362] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:37:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:37:58.186234 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836799b-513b-4da0-ba34-9b70564ec5f5 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:37:58.189202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b836799b-513b-4da0-ba34-9b70564ec5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:58.455455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836799b-513b-4da0-ba34-9b70564ec5f5 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:37:58.475944 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:37:58.691161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae3afb-d986-4eaf-88d3-0be18e6b0a5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:37:58.693086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae3afb-d986-4eaf-88d3-0be18e6b0a5d service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:37:58.693673 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae3afb-d986-4eaf-88d3-0be18e6b0a5d service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:37:58.694484 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae3afb-d986-4eaf-88d3-0be18e6b0a5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:37:58.699828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a0ae3afb-d986-4eaf-88d3-0be18e6b0a5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:37:58.871486 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae3afb-d986-4eaf-88d3-0be18e6b0a5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:37:58.916980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae3afb-d986-4eaf-88d3-0be18e6b0a5d service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:37:58.916980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 679/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:37:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 725 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:38:02.003723 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d2dbb-b487-4aa0-bd09-5d56784ad27f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:02.004379 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d2dbb-b487-4aa0-bd09-5d56784ad27f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:02.004379 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d2dbb-b487-4aa0-bd09-5d56784ad27f None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:02.005906 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 685/1364] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:38:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:38:02.012524 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef83d7f-08f9-4b35-86fa-fc307aac8cda None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:02.013000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef83d7f-08f9-4b35-86fa-fc307aac8cda None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:02.013211 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef83d7f-08f9-4b35-86fa-fc307aac8cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:02.028702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:02.108092 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-fef83d7f-08f9-4b35-86fa-fc307aac8cda None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:02.109268 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-fef83d7f-08f9-4b35-86fa-fc307aac8cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:02.252339 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 680/1365] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:38:02.326563 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506e68e-af3e-4b1f-bcd2-891333192ed4 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:02.326855 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506e68e-af3e-4b1f-bcd2-891333192ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:02.327106 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506e68e-af3e-4b1f-bcd2-891333192ed4 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:02.328700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 686/1366] 10.222.0.57 () {54 vars in 962 bytes} [Mon Jul 18 08:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:38:02.336062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b45e2b-14e0-40df-95a2-489949c47284 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:02.336345 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b45e2b-14e0-40df-95a2-489949c47284 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:02.336633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b45e2b-14e0-40df-95a2-489949c47284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:02.441952 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-12b45e2b-14e0-40df-95a2-489949c47284 None None] MFA Rules not processed for user `84cba474fc7c470ea119de697b9d7ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:02.443240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-12b45e2b-14e0-40df-95a2-489949c47284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:02.488077 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:02.591224 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 681/1367] 10.222.0.57 () {58 vars in 1059 bytes} [Mon Jul 18 08:38:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:38:02.599178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506e68e-af3e-4b1f-bcd2-891333192ed4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:38:02.600729 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c506e68e-af3e-4b1f-bcd2-891333192ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:02.766780 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506e68e-af3e-4b1f-bcd2-891333192ed4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:38:02.789074 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:02.906430 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6893fa-b3bc-40ac-afcf-594b4bb11c9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:38:02.907179 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6893fa-b3bc-40ac-afcf-594b4bb11c9f service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:02.907391 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6893fa-b3bc-40ac-afcf-594b4bb11c9f service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:02.907641 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6893fa-b3bc-40ac-afcf-594b4bb11c9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:02.909458 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-dc6893fa-b3bc-40ac-afcf-594b4bb11c9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:03.025202 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6893fa-b3bc-40ac-afcf-594b4bb11c9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:38:03.028561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6893fa-b3bc-40ac-afcf-594b4bb11c9f service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:38:03.050927 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 687/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:38:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:38:04.856549 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6746d17-2efe-4e7f-8b3a-3b3baaac5dc7 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:04.857184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6746d17-2efe-4e7f-8b3a-3b3baaac5dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:04.857184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6746d17-2efe-4e7f-8b3a-3b3baaac5dc7 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:04.858713 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 682/1369] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:38:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:38:04.864567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1d8d7-ced1-400a-bf83-ae680907b1e5 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:04.865110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1d8d7-ced1-400a-bf83-ae680907b1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:04.865323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1d8d7-ced1-400a-bf83-ae680907b1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:04.907047 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:04.950611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a3f1d8d7-ced1-400a-bf83-ae680907b1e5 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:04.951921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a3f1d8d7-ced1-400a-bf83-ae680907b1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:05.079721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 688/1370] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:38:05.396925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1738a1-8144-4898-a951-48bb70d88bfa None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:05.397307 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1738a1-8144-4898-a951-48bb70d88bfa None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:05.397495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1738a1-8144-4898-a951-48bb70d88bfa None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:05.399037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 683/1371] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:38:05.405104 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2586786-f74b-480e-883a-675953c5ff68 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:05.405428 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2586786-f74b-480e-883a-675953c5ff68 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:05.405687 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2586786-f74b-480e-883a-675953c5ff68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:05.486199 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a2586786-f74b-480e-883a-675953c5ff68 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:05.487438 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a2586786-f74b-480e-883a-675953c5ff68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:05.512366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfd0f7-31d1-46ce-8bf4-4985fcebb2dc None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:05.512748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfd0f7-31d1-46ce-8bf4-4985fcebb2dc None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:05.513145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfd0f7-31d1-46ce-8bf4-4985fcebb2dc None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:05.514718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 684/1372] 10.222.0.57 () {54 vars in 964 bytes} [Mon Jul 18 08:38:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:38:05.519887 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91976f-d513-476e-934d-e3068e8b85cd None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:05.520179 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91976f-d513-476e-934d-e3068e8b85cd None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:05.520461 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91976f-d513-476e-934d-e3068e8b85cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:05.550199 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:05.613193 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 689/1373] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:38:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:38:05.621593 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-bc91976f-d513-476e-934d-e3068e8b85cd None None] MFA Rules not processed for user `d5f9a129021445ae8154a5e6941291e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:05.625764 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bc91976f-d513-476e-934d-e3068e8b85cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:05.660775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2586786-f74b-480e-883a-675953c5ff68 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:38:05.662212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a2586786-f74b-480e-883a-675953c5ff68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:05.735350 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 685/1374] 10.222.0.57 () {58 vars in 1061 bytes} [Mon Jul 18 08:38:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:38:05.743488 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91976f-d513-476e-934d-e3068e8b85cd None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:38:05.754718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91976f-d513-476e-934d-e3068e8b85cd None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:38:05.756588 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:05.832984 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2586786-f74b-480e-883a-675953c5ff68 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:38:05.867150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ad5ce-82f5-4e5a-bd0b-888e7881206c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:38:05.868433 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ad5ce-82f5-4e5a-bd0b-888e7881206c service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:05.868891 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ad5ce-82f5-4e5a-bd0b-888e7881206c service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:05.869350 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ad5ce-82f5-4e5a-bd0b-888e7881206c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:05.872556 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e47ad5ce-82f5-4e5a-bd0b-888e7881206c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:05.957744 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:06.053901 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed18546-48ec-41df-87e4-682b760d9b32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:38:06.054333 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed18546-48ec-41df-87e4-682b760d9b32 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:06.054620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed18546-48ec-41df-87e4-682b760d9b32 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:06.055555 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed18546-48ec-41df-87e4-682b760d9b32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:06.057022 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-eed18546-48ec-41df-87e4-682b760d9b32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:06.064114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47ad5ce-82f5-4e5a-bd0b-888e7881206c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:38:06.065629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47ad5ce-82f5-4e5a-bd0b-888e7881206c service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:38:06.086450 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 686/1375] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:38:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:38:06.164413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed18546-48ec-41df-87e4-682b760d9b32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:38:06.166051 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed18546-48ec-41df-87e4-682b760d9b32 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:38:06.188234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 690/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:38:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:38:08.947298 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325d321-1ae5-4b9f-bf4f-c28a5a45b141 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:08.947852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325d321-1ae5-4b9f-bf4f-c28a5a45b141 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:08.948367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325d321-1ae5-4b9f-bf4f-c28a5a45b141 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:08.950955 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 687/1377] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:38:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:38:08.956992 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d730aa-99b0-4c4a-837d-9d8d2fa0dcd6 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:08.957760 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d730aa-99b0-4c4a-837d-9d8d2fa0dcd6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:08.958045 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d730aa-99b0-4c4a-837d-9d8d2fa0dcd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:09.061508 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c9d730aa-99b0-4c4a-837d-9d8d2fa0dcd6 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:09.062973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c9d730aa-99b0-4c4a-837d-9d8d2fa0dcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:09.099134 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:09.178097 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 691/1378] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:38:09.232861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325d321-1ae5-4b9f-bf4f-c28a5a45b141 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:38:09.234471 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1325d321-1ae5-4b9f-bf4f-c28a5a45b141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:09.274831 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:09.405831 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325d321-1ae5-4b9f-bf4f-c28a5a45b141 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:38:09.598675 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:09.607656 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc0004-9431-413f-b58f-23a38a63cf5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:38:09.608350 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc0004-9431-413f-b58f-23a38a63cf5b service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:09.608632 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc0004-9431-413f-b58f-23a38a63cf5b service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:09.608905 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc0004-9431-413f-b58f-23a38a63cf5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:09.610671 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-45cc0004-9431-413f-b58f-23a38a63cf5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:09.706644 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc0004-9431-413f-b58f-23a38a63cf5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:38:09.708301 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc0004-9431-413f-b58f-23a38a63cf5b service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:38:09.727429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 688/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:38:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:38:12.286267 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9f1ff-f39a-4418-9539-4374f04f128b None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:12.286789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9f1ff-f39a-4418-9539-4374f04f128b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:12.286789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9f1ff-f39a-4418-9539-4374f04f128b None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:12.288311 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 692/1380] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:38:12.293733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dee20e-d333-4c41-be2d-5ad560d95130 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:12.294297 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dee20e-d333-4c41-be2d-5ad560d95130 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:12.294297 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dee20e-d333-4c41-be2d-5ad560d95130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:12.377454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-32dee20e-d333-4c41-be2d-5ad560d95130 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:12.378897 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-32dee20e-d333-4c41-be2d-5ad560d95130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:12.458716 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:12.523761 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 689/1381] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:38:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:38:12.571721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9f1ff-f39a-4418-9539-4374f04f128b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:38:12.574162 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-44e9f1ff-f39a-4418-9539-4374f04f128b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:12.747022 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9f1ff-f39a-4418-9539-4374f04f128b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:38:12.759735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:12.879062 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa116a71-f84f-4527-97f1-f360b5bb6a32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:38:12.881019 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa116a71-f84f-4527-97f1-f360b5bb6a32 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:12.881019 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa116a71-f84f-4527-97f1-f360b5bb6a32 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:12.881019 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa116a71-f84f-4527-97f1-f360b5bb6a32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:12.882069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-aa116a71-f84f-4527-97f1-f360b5bb6a32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:13.004205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa116a71-f84f-4527-97f1-f360b5bb6a32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:38:13.005865 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa116a71-f84f-4527-97f1-f360b5bb6a32 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:38:13.030975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 693/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:38:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:38:15.422124 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850c664-e52c-49bf-92dd-5696266436b2 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:15.422906 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850c664-e52c-49bf-92dd-5696266436b2 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:15.423050 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850c664-e52c-49bf-92dd-5696266436b2 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:15.425534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 690/1383] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:38:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:38:15.434552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5a1ba-3d12-4d01-ad99-e51bf07b65b6 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:15.435075 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5a1ba-3d12-4d01-ad99-e51bf07b65b6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:15.435485 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5a1ba-3d12-4d01-ad99-e51bf07b65b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:15.464383 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:15.544581 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-12a5a1ba-3d12-4d01-ad99-e51bf07b65b6 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:15.545875 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-12a5a1ba-3d12-4d01-ad99-e51bf07b65b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:15.676519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 694/1384] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:38:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:38:15.726491 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850c664-e52c-49bf-92dd-5696266436b2 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:38:15.728239 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5850c664-e52c-49bf-92dd-5696266436b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:15.938933 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850c664-e52c-49bf-92dd-5696266436b2 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:38:16.007314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:16.093418 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdea86-f8fb-4c39-9e29-23162fb9c6c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:38:16.099867 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdea86-f8fb-4c39-9e29-23162fb9c6c6 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:16.099867 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdea86-f8fb-4c39-9e29-23162fb9c6c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:16.099867 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdea86-f8fb-4c39-9e29-23162fb9c6c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:16.099867 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e9bdea86-f8fb-4c39-9e29-23162fb9c6c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:16.217839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bdea86-f8fb-4c39-9e29-23162fb9c6c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:38:16.219321 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bdea86-f8fb-4c39-9e29-23162fb9c6c6 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:38:16.247738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 691/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:38:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:38:18.788252 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b23441-b965-464a-b6be-9dd0b8cc5922 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:18.788856 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b23441-b965-464a-b6be-9dd0b8cc5922 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:18.788856 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b23441-b965-464a-b6be-9dd0b8cc5922 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:18.790246 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 695/1386] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:38:18.797398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f327b-b68f-4b6d-b6d2-f20f642d4c88 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:18.798109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f327b-b68f-4b6d-b6d2-f20f642d4c88 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:18.798109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f327b-b68f-4b6d-b6d2-f20f642d4c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:18.895210 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-eb3f327b-b68f-4b6d-b6d2-f20f642d4c88 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:18.897166 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-eb3f327b-b68f-4b6d-b6d2-f20f642d4c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:18.908289 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:19.014419 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 692/1387] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:38:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:38:19.063275 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b23441-b965-464a-b6be-9dd0b8cc5922 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:38:19.064884 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e3b23441-b965-464a-b6be-9dd0b8cc5922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:19.193824 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:19.273475 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b23441-b965-464a-b6be-9dd0b8cc5922 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:38:19.438141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9a961-5163-4e9c-acb9-e1db3db98294 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:38:19.438789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9a961-5163-4e9c-acb9-e1db3db98294 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:19.439128 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9a961-5163-4e9c-acb9-e1db3db98294 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:19.439435 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9a961-5163-4e9c-acb9-e1db3db98294 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:19.441243 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-51a9a961-5163-4e9c-acb9-e1db3db98294 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:19.518931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:19.566502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9a961-5163-4e9c-acb9-e1db3db98294 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:38:19.570217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9a961-5163-4e9c-acb9-e1db3db98294 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:38:19.598364 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 696/1388] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:38:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:38:22.054046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd03dc6-63c9-4ff3-844f-3e190beb27ce None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:22.054634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd03dc6-63c9-4ff3-844f-3e190beb27ce None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:22.054634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd03dc6-63c9-4ff3-844f-3e190beb27ce None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:22.056183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 693/1389] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:38:22.062411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058b9e3-a17b-4b74-8f20-45e1f5310da3 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:22.062910 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058b9e3-a17b-4b74-8f20-45e1f5310da3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:22.063116 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058b9e3-a17b-4b74-8f20-45e1f5310da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:22.144111 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0058b9e3-a17b-4b74-8f20-45e1f5310da3 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:22.145548 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0058b9e3-a17b-4b74-8f20-45e1f5310da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:22.240982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15e939-7a85-45ab-9781-a11f6f4cfaea None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:22.241453 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15e939-7a85-45ab-9781-a11f6f4cfaea None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:22.241914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15e939-7a85-45ab-9781-a11f6f4cfaea None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:22.245512 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 694/1390] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:38:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:38:22.254074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd3bbd-3f56-4ac4-947a-59dc57877814 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:22.254445 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd3bbd-3f56-4ac4-947a-59dc57877814 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:22.254870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd3bbd-3f56-4ac4-947a-59dc57877814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:22.267076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 697/1391] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:38:22.317919 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058b9e3-a17b-4b74-8f20-45e1f5310da3 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:38:22.319304 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0058b9e3-a17b-4b74-8f20-45e1f5310da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:22.339653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c0cd3bbd-3f56-4ac4-947a-59dc57877814 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:22.341134 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c0cd3bbd-3f56-4ac4-947a-59dc57877814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:22.363224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:22.444219 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:22.452438 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058b9e3-a17b-4b74-8f20-45e1f5310da3 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:38:22.458852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 695/1392] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:38:22.520221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b2441-cbfb-4a4f-9cf3-652eba7b7424 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:22.520583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b2441-cbfb-4a4f-9cf3-652eba7b7424 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:22.521004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b2441-cbfb-4a4f-9cf3-652eba7b7424 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:22.522939 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 696/1393] 10.222.0.57 () {54 vars in 961 bytes} [Mon Jul 18 08:38:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:38:22.529580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e03348-c57d-4a27-a24a-94aa23f45d2c None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:22.529936 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e03348-c57d-4a27-a24a-94aa23f45d2c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:22.530348 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e03348-c57d-4a27-a24a-94aa23f45d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:22.586317 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec0923-feb5-4585-96c4-d04a94f11537 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:38:22.586975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec0923-feb5-4585-96c4-d04a94f11537 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:22.587255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec0923-feb5-4585-96c4-d04a94f11537 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:22.587531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec0923-feb5-4585-96c4-d04a94f11537 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:22.591607 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-31ec0923-feb5-4585-96c4-d04a94f11537 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:22.626571 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:22.641311 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-96e03348-c57d-4a27-a24a-94aa23f45d2c None None] MFA Rules not processed for user `694088ed30b94bad9d3b16f3766252df`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:22.648322 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-96e03348-c57d-4a27-a24a-94aa23f45d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:22.732322 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 697/1394] 10.222.0.57 () {58 vars in 1058 bytes} [Mon Jul 18 08:38:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:38:22.733100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec0923-feb5-4585-96c4-d04a94f11537 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:38:22.735856 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec0923-feb5-4585-96c4-d04a94f11537 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:38:22.740243 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e03348-c57d-4a27-a24a-94aa23f45d2c None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:38:22.750206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e03348-c57d-4a27-a24a-94aa23f45d2c None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:38:22.770039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 698/1395] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:38:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:38:22.839878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2580f82-cf0d-426e-95f1-6fdeef486a80 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '694088ed30b94bad9d3b16f3766252df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:38:22.841006 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2580f82-cf0d-426e-95f1-6fdeef486a80 service glance] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:22.841006 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2580f82-cf0d-426e-95f1-6fdeef486a80 service glance] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:22.841230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2580f82-cf0d-426e-95f1-6fdeef486a80 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:22.842782 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d2580f82-cf0d-426e-95f1-6fdeef486a80 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:22.845813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:22.969327 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2580f82-cf0d-426e-95f1-6fdeef486a80 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:38:22.971075 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2580f82-cf0d-426e-95f1-6fdeef486a80 service glance] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:38:22.994970 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 698/1396] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:38:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:38:23.917989 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce898770-e0a1-40eb-8264-9f735782b46f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:23.919207 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce898770-e0a1-40eb-8264-9f735782b46f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:23.919207 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce898770-e0a1-40eb-8264-9f735782b46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:24.032718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ce898770-e0a1-40eb-8264-9f735782b46f None None] MFA Rules not processed for user `2b03f8b8c51e472bbd4cf0afc9c6b632`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:24.034006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ce898770-e0a1-40eb-8264-9f735782b46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:24.147423 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 699/1397] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Jul 18 08:38:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:38:24.205175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55a81c-54e6-4ba6-b095-6ae163930248 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:24.205733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55a81c-54e6-4ba6-b095-6ae163930248 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:24.205911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55a81c-54e6-4ba6-b095-6ae163930248 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:24.207182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 699/1398] 10.222.0.57 () {54 vars in 943 bytes} [Mon Jul 18 08:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:38:24.216710 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586c02c-5067-47fe-8646-cae372c5a0ed None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:24.217178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586c02c-5067-47fe-8646-cae372c5a0ed None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:24.217428 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586c02c-5067-47fe-8646-cae372c5a0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:24.248444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:24.335842 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e586c02c-5067-47fe-8646-cae372c5a0ed None None] MFA Rules not processed for user `09a5c2e804504038817cad76544dce5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:24.337328 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e586c02c-5067-47fe-8646-cae372c5a0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:24.473743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 700/1399] 10.222.0.57 () {58 vars in 1040 bytes} [Mon Jul 18 08:38:24 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:38:24.486156 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55a81c-54e6-4ba6-b095-6ae163930248 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:38:24.487837 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-fa55a81c-54e6-4ba6-b095-6ae163930248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:24.617729 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:24.659219 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55a81c-54e6-4ba6-b095-6ae163930248 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:38:24.787933 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e6800-8a81-4253-9627-1ab198346edc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09a5c2e804504038817cad76544dce5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:38:24.788907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e6800-8a81-4253-9627-1ab198346edc service swift] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:24.789394 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e6800-8a81-4253-9627-1ab198346edc service swift] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:24.789884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e6800-8a81-4253-9627-1ab198346edc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:24.793572 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-652e6800-8a81-4253-9627-1ab198346edc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:24.920582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e6800-8a81-4253-9627-1ab198346edc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:38:24.922523 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e6800-8a81-4253-9627-1ab198346edc service swift] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:38:24.932699 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 700/1400] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Jul 18 08:38:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 08:38:25.245171 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d4c8c-21e9-4ac2-9e5a-64e66408a7ed None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:25.245764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d4c8c-21e9-4ac2-9e5a-64e66408a7ed None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:25.245764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d4c8c-21e9-4ac2-9e5a-64e66408a7ed None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:25.247111 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 701/1401] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:38:25.252791 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade9b82-43c2-4b2b-b650-f383fe498950 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:25.253318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade9b82-43c2-4b2b-b650-f383fe498950 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:25.253435 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade9b82-43c2-4b2b-b650-f383fe498950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:25.257544 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:25.334085 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-7ade9b82-43c2-4b2b-b650-f383fe498950 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:25.335904 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7ade9b82-43c2-4b2b-b650-f383fe498950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:25.468917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 701/1402] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:38:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:38:25.522746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d4c8c-21e9-4ac2-9e5a-64e66408a7ed None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:38:25.524595 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d57d4c8c-21e9-4ac2-9e5a-64e66408a7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:25.629366 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:25.715169 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d4c8c-21e9-4ac2-9e5a-64e66408a7ed None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:38:25.868469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b2cee-2520-41f6-a905-ab74dbca2fbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:38:25.869617 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b2cee-2520-41f6-a905-ab74dbca2fbd service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:25.869931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b2cee-2520-41f6-a905-ab74dbca2fbd service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:25.870273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b2cee-2520-41f6-a905-ab74dbca2fbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:25.872403 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b41b2cee-2520-41f6-a905-ab74dbca2fbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:25.934835 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:26.004236 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b2cee-2520-41f6-a905-ab74dbca2fbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:38:26.008384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b2cee-2520-41f6-a905-ab74dbca2fbd service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:38:26.028539 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 702/1403] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:38:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:38:28.396834 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66993be-a901-40ed-b0e1-c86b38f2a639 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:28.397484 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66993be-a901-40ed-b0e1-c86b38f2a639 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:28.397484 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66993be-a901-40ed-b0e1-c86b38f2a639 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:28.399785 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 702/1404] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:38:28.405220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bbbf0-ae3d-40b0-87b1-5d066080f0fa None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:28.405738 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bbbf0-ae3d-40b0-87b1-5d066080f0fa None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:28.405842 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bbbf0-ae3d-40b0-87b1-5d066080f0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:28.511941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-724bbbf0-ae3d-40b0-87b1-5d066080f0fa None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:28.513246 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-724bbbf0-ae3d-40b0-87b1-5d066080f0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:28.646674 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 703/1405] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:38:28.703183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66993be-a901-40ed-b0e1-c86b38f2a639 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:38:28.705037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d66993be-a901-40ed-b0e1-c86b38f2a639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:28.818691 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:28.908267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66993be-a901-40ed-b0e1-c86b38f2a639 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:38:29.061801 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a9c43-d271-412b-aa4b-397ecbccbd0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:38:29.062441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a9c43-d271-412b-aa4b-397ecbccbd0b service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:29.062791 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a9c43-d271-412b-aa4b-397ecbccbd0b service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:29.063068 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a9c43-d271-412b-aa4b-397ecbccbd0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:29.064934 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c68a9c43-d271-412b-aa4b-397ecbccbd0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:29.121807 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:29.183869 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a9c43-d271-412b-aa4b-397ecbccbd0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:38:29.185587 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a9c43-d271-412b-aa4b-397ecbccbd0b service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:38:29.213010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 703/1406] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:38:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:38:31.843965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40257f36-643d-4d54-a39b-320c9ca13695 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:31.844460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40257f36-643d-4d54-a39b-320c9ca13695 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:31.844554 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40257f36-643d-4d54-a39b-320c9ca13695 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:31.846327 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 704/1407] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:38:31.851894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3daf01-5623-476b-bec5-a6c5393e8de4 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:31.852363 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3daf01-5623-476b-bec5-a6c5393e8de4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:31.852474 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3daf01-5623-476b-bec5-a6c5393e8de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:31.925190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-5b3daf01-5623-476b-bec5-a6c5393e8de4 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:31.926419 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5b3daf01-5623-476b-bec5-a6c5393e8de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:32.044415 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 704/1408] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:38:32.096128 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40257f36-643d-4d54-a39b-320c9ca13695 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:38:32.096128 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-40257f36-643d-4d54-a39b-320c9ca13695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:32.117949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:32.291615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40257f36-643d-4d54-a39b-320c9ca13695 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:38:32.419810 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:32.460721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc600b-e161-47e7-a65f-4483bbd46179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:38:32.461518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc600b-e161-47e7-a65f-4483bbd46179 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:32.461859 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc600b-e161-47e7-a65f-4483bbd46179 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:32.462227 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc600b-e161-47e7-a65f-4483bbd46179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:32.464478 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5ccc600b-e161-47e7-a65f-4483bbd46179 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:32.633803 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc600b-e161-47e7-a65f-4483bbd46179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:38:32.636083 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc600b-e161-47e7-a65f-4483bbd46179 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:38:32.657023 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 705/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:38:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:38:42.041739 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:42.721074 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=55 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:50.269764 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234c4a3-d2a3-4f63-ab70-85e2e1fdf284 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:50.270302 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234c4a3-d2a3-4f63-ab70-85e2e1fdf284 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:50.270302 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234c4a3-d2a3-4f63-ab70-85e2e1fdf284 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:50.271791 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 705/1410] 10.222.0.57 () {54 vars in 918 bytes} [Mon Jul 18 08:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:38:50.278301 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457f8d9-344c-4c94-b817-a19c845eb723 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:50.278877 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457f8d9-344c-4c94-b817-a19c845eb723 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:50.279059 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457f8d9-344c-4c94-b817-a19c845eb723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:50.359412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b457f8d9-344c-4c94-b817-a19c845eb723 None None] MFA Rules not processed for user `d606fd7ef269460a8110223932f02d19`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:50.361055 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b457f8d9-344c-4c94-b817-a19c845eb723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:50.503240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 706/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Mon Jul 18 08:38:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:38:50.518296 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9883f33-25d0-45fd-bed1-30abca2fa694 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:50.518589 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9883f33-25d0-45fd-bed1-30abca2fa694 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:50.518868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9883f33-25d0-45fd-bed1-30abca2fa694 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:50.520292 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 706/1412] 10.222.0.57 () {54 vars in 918 bytes} [Mon Jul 18 08:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:38:50.525956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36250d7b-d950-4b5c-ab99-4b3798b5c798 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:50.526247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36250d7b-d950-4b5c-ab99-4b3798b5c798 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:50.526563 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36250d7b-d950-4b5c-ab99-4b3798b5c798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:50.602369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:50.605773 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-36250d7b-d950-4b5c-ab99-4b3798b5c798 None None] MFA Rules not processed for user `84cba474fc7c470ea119de697b9d7ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:50.607002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-36250d7b-d950-4b5c-ab99-4b3798b5c798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:50.729396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 707/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Mon Jul 18 08:38:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:38:53.462367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b151d-2911-4e3e-bd81-1034530358a9 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:53.462963 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b151d-2911-4e3e-bd81-1034530358a9 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:53.462963 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b151d-2911-4e3e-bd81-1034530358a9 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:53.464437 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 08:38:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:38:53.470404 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b5d54-84d9-4fdb-8464-d299863ac299 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:53.470860 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b5d54-84d9-4fdb-8464-d299863ac299 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:53.470964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b5d54-84d9-4fdb-8464-d299863ac299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:53.539931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b25b5d54-84d9-4fdb-8464-d299863ac299 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:53.541191 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b25b5d54-84d9-4fdb-8464-d299863ac299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:53.597799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:53.641240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 08:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:38:53.685809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b151d-2911-4e3e-bd81-1034530358a9 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:38:53.687346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-dd8b151d-2911-4e3e-bd81-1034530358a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:53.842462 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b151d-2911-4e3e-bd81-1034530358a9 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:38:53.917458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:53.959740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda547a-e749-4a72-b717-81408d1dbd41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:38:53.960396 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda547a-e749-4a72-b717-81408d1dbd41 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:53.960682 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda547a-e749-4a72-b717-81408d1dbd41 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:53.960977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda547a-e749-4a72-b717-81408d1dbd41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:53.963297 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3cda547a-e749-4a72-b717-81408d1dbd41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:54.053545 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda547a-e749-4a72-b717-81408d1dbd41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:38:54.055154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda547a-e749-4a72-b717-81408d1dbd41 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:38:54.073464 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 708/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:38:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:38:55.003053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d688d3-97a6-4c25-939c-020177024146 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:55.003709 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d688d3-97a6-4c25-939c-020177024146 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:55.003709 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d688d3-97a6-4c25-939c-020177024146 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:55.005020 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 08:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:38:55.010819 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0e48a-9cc8-485d-88a7-bd39aff47f8d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:55.011308 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0e48a-9cc8-485d-88a7-bd39aff47f8d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:55.011420 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0e48a-9cc8-485d-88a7-bd39aff47f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:55.095411 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c9b0e48a-9cc8-485d-88a7-bd39aff47f8d None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:55.096583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c9b0e48a-9cc8-485d-88a7-bd39aff47f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:55.127692 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:55.203113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 08:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:38:55.359853 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d688d3-97a6-4c25-939c-020177024146 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:38:55.361289 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-16d688d3-97a6-4c25-939c-020177024146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:55.517618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d688d3-97a6-4c25-939c-020177024146 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:38:55.562207 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:55.643925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3a0f3-e5bc-438c-b7f1-eec1acf9413f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:38:55.644485 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3a0f3-e5bc-438c-b7f1-eec1acf9413f service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:55.644777 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3a0f3-e5bc-438c-b7f1-eec1acf9413f service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:55.645059 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3a0f3-e5bc-438c-b7f1-eec1acf9413f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:55.646857 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f6e3a0f3-e5bc-438c-b7f1-eec1acf9413f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:55.746409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3a0f3-e5bc-438c-b7f1-eec1acf9413f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:38:55.748087 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3a0f3-e5bc-438c-b7f1-eec1acf9413f service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:38:55.768527 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 710/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:38:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:38:56.678648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d349e0-ce4a-47ee-9489-9ef50de00d8c None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:56.679222 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d349e0-ce4a-47ee-9489-9ef50de00d8c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:56.679222 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d349e0-ce4a-47ee-9489-9ef50de00d8c None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:56.680701 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 08:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:38:56.690044 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c846f-cfa5-4e65-9f23-146a79420048 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:56.690624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c846f-cfa5-4e65-9f23-146a79420048 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:56.690624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c846f-cfa5-4e65-9f23-146a79420048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:56.751677 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:56.775557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f84c846f-cfa5-4e65-9f23-146a79420048 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:56.776791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f84c846f-cfa5-4e65-9f23-146a79420048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:56.882883 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 08:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:38:57.034482 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d349e0-ce4a-47ee-9489-9ef50de00d8c None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:38:57.036079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-73d349e0-ce4a-47ee-9489-9ef50de00d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:57.188696 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d349e0-ce4a-47ee-9489-9ef50de00d8c None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:38:57.201820 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:57.427104 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec135064-6faf-4e99-8cc2-cdb68fdcabc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:38:57.427706 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec135064-6faf-4e99-8cc2-cdb68fdcabc4 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:57.427995 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec135064-6faf-4e99-8cc2-cdb68fdcabc4 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:57.428265 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec135064-6faf-4e99-8cc2-cdb68fdcabc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:57.430624 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ec135064-6faf-4e99-8cc2-cdb68fdcabc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:57.524899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec135064-6faf-4e99-8cc2-cdb68fdcabc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:38:57.526510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec135064-6faf-4e99-8cc2-cdb68fdcabc4 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:38:57.545406 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 711/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:38:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:38:58.418357 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636f075-29dc-4ca8-9faf-b05e10a2ecd5 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:58.418921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636f075-29dc-4ca8-9faf-b05e10a2ecd5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:58.418921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636f075-29dc-4ca8-9faf-b05e10a2ecd5 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:58.420294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 08:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:38:58.425232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d54f72-65dc-4c5e-a960-50bb12d787ed None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:58.425692 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d54f72-65dc-4c5e-a960-50bb12d787ed None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:58.425827 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d54f72-65dc-4c5e-a960-50bb12d787ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:58.444813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:58.502277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-62d54f72-65dc-4c5e-a960-50bb12d787ed None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:38:58.503476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-62d54f72-65dc-4c5e-a960-50bb12d787ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:58.607249 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 08:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:38:58.760196 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636f075-29dc-4ca8-9faf-b05e10a2ecd5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:38:58.761715 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1636f075-29dc-4ca8-9faf-b05e10a2ecd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:58.874197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:58.910927 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636f075-29dc-4ca8-9faf-b05e10a2ecd5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:38:59.027788 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0305da80-7e1e-4290-923e-dec7968d6b2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:38:59.028365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305da80-7e1e-4290-923e-dec7968d6b2b service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:38:59.028652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305da80-7e1e-4290-923e-dec7968d6b2b service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:38:59.028932 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305da80-7e1e-4290-923e-dec7968d6b2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:38:59.030604 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0305da80-7e1e-4290-923e-dec7968d6b2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:38:59.107972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:38:59.122251 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0305da80-7e1e-4290-923e-dec7968d6b2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:38:59.123925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0305da80-7e1e-4290-923e-dec7968d6b2b service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:38:59.142931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 713/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:38:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:39:00.039475 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9541c7-2458-4771-93a2-357e5e178282 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:00.040098 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9541c7-2458-4771-93a2-357e5e178282 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:00.040280 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9541c7-2458-4771-93a2-357e5e178282 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:00.041720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 08:39:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:39:00.047552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7eac77-a450-445b-9662-c6a296e651ad None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:00.048130 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7eac77-a450-445b-9662-c6a296e651ad None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:00.048224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7eac77-a450-445b-9662-c6a296e651ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:00.134234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0c7eac77-a450-445b-9662-c6a296e651ad None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:39:00.135469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0c7eac77-a450-445b-9662-c6a296e651ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:00.239325 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 08:39:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:39:00.391878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9541c7-2458-4771-93a2-357e5e178282 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:39:00.393191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0f9541c7-2458-4771-93a2-357e5e178282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:00.482242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:00.548794 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9541c7-2458-4771-93a2-357e5e178282 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:39:00.667560 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65069c5a-6ba1-4de8-85b5-bf1471612f85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:39:00.668136 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65069c5a-6ba1-4de8-85b5-bf1471612f85 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:00.668411 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65069c5a-6ba1-4de8-85b5-bf1471612f85 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:00.668677 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65069c5a-6ba1-4de8-85b5-bf1471612f85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:00.670297 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-65069c5a-6ba1-4de8-85b5-bf1471612f85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:00.718242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:00.760531 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65069c5a-6ba1-4de8-85b5-bf1471612f85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:39:00.762127 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65069c5a-6ba1-4de8-85b5-bf1471612f85 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:39:00.781337 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 714/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:39:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:39:01.667747 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e4046-368d-4b78-9c92-21dfa1f75423 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:01.668358 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e4046-368d-4b78-9c92-21dfa1f75423 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:01.668358 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e4046-368d-4b78-9c92-21dfa1f75423 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:01.669681 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 08:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:39:01.675009 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f459c-7e5b-4814-bc39-7815f30d8940 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:01.675431 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f459c-7e5b-4814-bc39-7815f30d8940 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:01.675566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f459c-7e5b-4814-bc39-7815f30d8940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:01.751991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c30f459c-7e5b-4814-bc39-7815f30d8940 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:39:01.753248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c30f459c-7e5b-4814-bc39-7815f30d8940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:01.851965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 08:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:39:02.003688 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e4046-368d-4b78-9c92-21dfa1f75423 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:39:02.005218 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fb1e4046-368d-4b78-9c92-21dfa1f75423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:02.056386 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:02.157992 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e4046-368d-4b78-9c92-21dfa1f75423 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:39:02.289955 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed05ca-54d4-4b52-bda4-e44690a73b86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:39:02.290608 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed05ca-54d4-4b52-bda4-e44690a73b86 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:02.290891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed05ca-54d4-4b52-bda4-e44690a73b86 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:02.291163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed05ca-54d4-4b52-bda4-e44690a73b86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:02.292851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e7ed05ca-54d4-4b52-bda4-e44690a73b86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:02.299867 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:02.385355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed05ca-54d4-4b52-bda4-e44690a73b86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:39:02.387012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed05ca-54d4-4b52-bda4-e44690a73b86 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:39:02.405491 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 716/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:39:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:39:03.290755 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b478805-20c7-4ca5-a76e-25b5d54cbd4b None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:03.291367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b478805-20c7-4ca5-a76e-25b5d54cbd4b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:03.291821 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b478805-20c7-4ca5-a76e-25b5d54cbd4b None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:03.294479 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 08:39:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:39:03.299878 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1238e6-ff43-4a5e-9473-0b711f9c5059 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:03.300395 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1238e6-ff43-4a5e-9473-0b711f9c5059 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:03.300496 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1238e6-ff43-4a5e-9473-0b711f9c5059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:03.381582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-4c1238e6-ff43-4a5e-9473-0b711f9c5059 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:39:03.383518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4c1238e6-ff43-4a5e-9473-0b711f9c5059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:03.452708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:03.482861 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 08:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:39:03.639003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b478805-20c7-4ca5-a76e-25b5d54cbd4b None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:39:03.640474 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6b478805-20c7-4ca5-a76e-25b5d54cbd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:03.658962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:03.802725 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b478805-20c7-4ca5-a76e-25b5d54cbd4b None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:39:03.903098 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:03.930665 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0cbc0-3d80-42bf-b6f2-66baf35a10df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:39:03.931322 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0cbc0-3d80-42bf-b6f2-66baf35a10df service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:03.931623 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0cbc0-3d80-42bf-b6f2-66baf35a10df service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:03.931920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0cbc0-3d80-42bf-b6f2-66baf35a10df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:03.933584 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-25a0cbc0-3d80-42bf-b6f2-66baf35a10df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:04.029339 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0cbc0-3d80-42bf-b6f2-66baf35a10df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:39:04.030923 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0cbc0-3d80-42bf-b6f2-66baf35a10df service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:39:04.049168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 717/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:39:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:39:04.950497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0eebd4-ecee-417b-9d8c-ed23484eaa2f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:04.951077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0eebd4-ecee-417b-9d8c-ed23484eaa2f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:04.951077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0eebd4-ecee-417b-9d8c-ed23484eaa2f None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:04.952442 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 08:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:39:04.957398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bf9d5-7ff8-4c97-99d0-1d68d284551f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:04.957867 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bf9d5-7ff8-4c97-99d0-1d68d284551f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:04.957966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bf9d5-7ff8-4c97-99d0-1d68d284551f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:05.037028 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-b96bf9d5-7ff8-4c97-99d0-1d68d284551f None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:39:05.038286 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b96bf9d5-7ff8-4c97-99d0-1d68d284551f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:05.082682 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:05.141139 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 08:39:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:39:05.299386 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0eebd4-ecee-417b-9d8c-ed23484eaa2f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:39:05.300975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1f0eebd4-ecee-417b-9d8c-ed23484eaa2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:05.455482 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0eebd4-ecee-417b-9d8c-ed23484eaa2f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:39:05.517602 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:05.585971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc78bd-3bcc-413f-9296-07c19bed2337 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:39:05.586639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc78bd-3bcc-413f-9296-07c19bed2337 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:05.586925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc78bd-3bcc-413f-9296-07c19bed2337 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:05.587323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc78bd-3bcc-413f-9296-07c19bed2337 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:05.589051 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-07dc78bd-3bcc-413f-9296-07c19bed2337 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:05.682190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc78bd-3bcc-413f-9296-07c19bed2337 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:39:05.683843 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc78bd-3bcc-413f-9296-07c19bed2337 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:39:05.702952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 719/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:39:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:39:06.591769 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585e384-cb4e-4533-b789-3fbf365ba75a None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:06.592377 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585e384-cb4e-4533-b789-3fbf365ba75a None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:06.592377 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585e384-cb4e-4533-b789-3fbf365ba75a None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:06.593989 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 08:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:39:06.599340 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c455496-ff79-42b7-9507-d0f2d516c57b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:06.599857 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c455496-ff79-42b7-9507-d0f2d516c57b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:06.599954 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c455496-ff79-42b7-9507-d0f2d516c57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:06.687268 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0c455496-ff79-42b7-9507-d0f2d516c57b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:39:06.688723 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0c455496-ff79-42b7-9507-d0f2d516c57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:06.691418 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:06.805326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 08:39:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:39:06.958250 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585e384-cb4e-4533-b789-3fbf365ba75a None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:39:06.959819 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f585e384-cb4e-4533-b789-3fbf365ba75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:07.117678 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585e384-cb4e-4533-b789-3fbf365ba75a None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:39:07.150261 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:07.241347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21fbf1-e6f1-41d0-b245-1cbb68664102 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:39:07.241903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21fbf1-e6f1-41d0-b245-1cbb68664102 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:07.242836 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21fbf1-e6f1-41d0-b245-1cbb68664102 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:07.243116 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21fbf1-e6f1-41d0-b245-1cbb68664102 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:07.244783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4f21fbf1-e6f1-41d0-b245-1cbb68664102 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:07.335067 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21fbf1-e6f1-41d0-b245-1cbb68664102 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:39:07.337478 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21fbf1-e6f1-41d0-b245-1cbb68664102 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:39:07.356386 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 720/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:39:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:39:09.308622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70f3c2-35b3-48df-b103-0c0ea0fe3ecb None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:09.309305 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70f3c2-35b3-48df-b103-0c0ea0fe3ecb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:09.309305 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70f3c2-35b3-48df-b103-0c0ea0fe3ecb None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:09.310540 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 08:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:39:09.316099 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ba359-174b-4322-853a-9786e4b644e5 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:09.316557 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ba359-174b-4322-853a-9786e4b644e5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:09.316673 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ba359-174b-4322-853a-9786e4b644e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:09.359640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:09.385249 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-890ba359-174b-4322-853a-9786e4b644e5 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:39:09.386631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-890ba359-174b-4322-853a-9786e4b644e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:09.485838 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 08:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:39:09.527911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70f3c2-35b3-48df-b103-0c0ea0fe3ecb None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:39:09.529528 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4d70f3c2-35b3-48df-b103-0c0ea0fe3ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:09.641478 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70f3c2-35b3-48df-b103-0c0ea0fe3ecb None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:39:09.674744 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:09.721203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128535ab-ef60-4bec-9c19-39009bb605a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:39:09.722029 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128535ab-ef60-4bec-9c19-39009bb605a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:09.722289 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128535ab-ef60-4bec-9c19-39009bb605a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:09.722566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128535ab-ef60-4bec-9c19-39009bb605a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:09.724449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-128535ab-ef60-4bec-9c19-39009bb605a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:09.838984 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128535ab-ef60-4bec-9c19-39009bb605a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:39:09.840466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128535ab-ef60-4bec-9c19-39009bb605a6 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:39:09.859891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 722/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:39:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:39:12.065869 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d1886-9876-4fd0-94d9-df4f1da04758 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:12.066444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d1886-9876-4fd0-94d9-df4f1da04758 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:12.066444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d1886-9876-4fd0-94d9-df4f1da04758 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:12.067833 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 08:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:39:12.073858 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec2d4d-1850-4b8d-bea4-27e30ef5d258 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:12.074319 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec2d4d-1850-4b8d-bea4-27e30ef5d258 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:12.074463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec2d4d-1850-4b8d-bea4-27e30ef5d258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:12.142157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b2ec2d4d-1850-4b8d-bea4-27e30ef5d258 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:39:12.143398 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b2ec2d4d-1850-4b8d-bea4-27e30ef5d258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:12.157254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:12.243948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 08:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:39:12.281998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d1886-9876-4fd0-94d9-df4f1da04758 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:39:12.283573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bc7d1886-9876-4fd0-94d9-df4f1da04758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:12.388108 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d1886-9876-4fd0-94d9-df4f1da04758 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:39:12.400850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:12.463497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6ef66-7ac9-4fdd-be6b-1b0ae34bf378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:39:12.464177 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6ef66-7ac9-4fdd-be6b-1b0ae34bf378 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:12.464466 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6ef66-7ac9-4fdd-be6b-1b0ae34bf378 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:12.464732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6ef66-7ac9-4fdd-be6b-1b0ae34bf378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:12.466373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2cc6ef66-7ac9-4fdd-be6b-1b0ae34bf378 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:12.582117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc6ef66-7ac9-4fdd-be6b-1b0ae34bf378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:39:12.583547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc6ef66-7ac9-4fdd-be6b-1b0ae34bf378 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:39:12.603339 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 723/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:39:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:39:15.115708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d63bd-e7e6-4896-b2df-59b123302800 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:15.116787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d63bd-e7e6-4896-b2df-59b123302800 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:15.116787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d63bd-e7e6-4896-b2df-59b123302800 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:15.118363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 08:39:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:39:15.124067 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8ffc3-b05d-470e-b97c-ddfaf62121dc None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:15.124528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8ffc3-b05d-470e-b97c-ddfaf62121dc None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:15.124623 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8ffc3-b05d-470e-b97c-ddfaf62121dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:15.172823 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:15.191853 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-53d8ffc3-b05d-470e-b97c-ddfaf62121dc None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:39:15.193082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-53d8ffc3-b05d-470e-b97c-ddfaf62121dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:15.294539 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 08:39:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:39:15.334063 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d63bd-e7e6-4896-b2df-59b123302800 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:39:15.336928 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-683d63bd-e7e6-4896-b2df-59b123302800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:15.442278 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d63bd-e7e6-4896-b2df-59b123302800 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:39:15.484390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:15.517988 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f5916-591a-451c-9da1-6c3b012d3e3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:39:15.518645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f5916-591a-451c-9da1-6c3b012d3e3e service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:15.518925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f5916-591a-451c-9da1-6c3b012d3e3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:15.519206 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f5916-591a-451c-9da1-6c3b012d3e3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:15.520924 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1f0f5916-591a-451c-9da1-6c3b012d3e3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:15.635064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f5916-591a-451c-9da1-6c3b012d3e3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:39:15.636553 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f5916-591a-451c-9da1-6c3b012d3e3e service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:39:15.657112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 725/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:39:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:39:17.817112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f22e26-4cab-4687-8085-1a381a5c9ff1 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:17.817709 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f22e26-4cab-4687-8085-1a381a5c9ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:17.817709 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f22e26-4cab-4687-8085-1a381a5c9ff1 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:17.819079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 08:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:39:17.825747 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894cf7e-4a7d-4750-9052-86c325cec498 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:17.826314 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894cf7e-4a7d-4750-9052-86c325cec498 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:17.826314 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894cf7e-4a7d-4750-9052-86c325cec498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:17.893819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-2894cf7e-4a7d-4750-9052-86c325cec498 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:39:17.895050 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2894cf7e-4a7d-4750-9052-86c325cec498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:17.917523 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:17.998000 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 08:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:39:18.037337 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f22e26-4cab-4687-8085-1a381a5c9ff1 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:39:18.038909 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-33f22e26-4cab-4687-8085-1a381a5c9ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:18.142514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f22e26-4cab-4687-8085-1a381a5c9ff1 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:39:18.165466 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:18.222320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca5267-01a3-43a1-a518-63e14fa5bf9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:39:18.222951 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca5267-01a3-43a1-a518-63e14fa5bf9e service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:18.223235 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca5267-01a3-43a1-a518-63e14fa5bf9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:18.223510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca5267-01a3-43a1-a518-63e14fa5bf9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:18.225197 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-68ca5267-01a3-43a1-a518-63e14fa5bf9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:18.344710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca5267-01a3-43a1-a518-63e14fa5bf9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:39:18.346145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca5267-01a3-43a1-a518-63e14fa5bf9e service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:39:18.364927 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 726/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:39:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:39:20.529037 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36404e1a-541b-458f-8e50-f141b0494888 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:20.529645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36404e1a-541b-458f-8e50-f141b0494888 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:20.529645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36404e1a-541b-458f-8e50-f141b0494888 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:20.530962 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 08:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:39:20.536768 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4eb37-0492-4d03-a54d-6b20f33603b4 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:20.537222 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4eb37-0492-4d03-a54d-6b20f33603b4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:20.537375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4eb37-0492-4d03-a54d-6b20f33603b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:20.603842 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:20.606463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-15c4eb37-0492-4d03-a54d-6b20f33603b4 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:39:20.607720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-15c4eb37-0492-4d03-a54d-6b20f33603b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:20.712248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 08:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:39:20.752211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36404e1a-541b-458f-8e50-f141b0494888 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:39:20.753750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-36404e1a-541b-458f-8e50-f141b0494888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:20.860513 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36404e1a-541b-458f-8e50-f141b0494888 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:39:20.909708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:20.935858 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabc8ef-52ee-49c6-9b0e-842a91b07cf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:39:20.936510 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabc8ef-52ee-49c6-9b0e-842a91b07cf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:20.936785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabc8ef-52ee-49c6-9b0e-842a91b07cf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:20.937056 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabc8ef-52ee-49c6-9b0e-842a91b07cf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:20.938680 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-dfabc8ef-52ee-49c6-9b0e-842a91b07cf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:21.056914 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfabc8ef-52ee-49c6-9b0e-842a91b07cf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:39:21.058415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfabc8ef-52ee-49c6-9b0e-842a91b07cf1 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:39:21.079698 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 728/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:39:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:39:23.191174 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1d99c-6a8a-4c30-a387-dbbad470a2cb None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:23.191746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1d99c-6a8a-4c30-a387-dbbad470a2cb None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:23.191856 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1d99c-6a8a-4c30-a387-dbbad470a2cb None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:23.193208 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 08:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:39:23.199127 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a726b1-0819-4dd9-bedc-850a106b1cbe None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:23.199556 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a726b1-0819-4dd9-bedc-850a106b1cbe None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:23.199713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a726b1-0819-4dd9-bedc-850a106b1cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:23.265870 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f5a726b1-0819-4dd9-bedc-850a106b1cbe None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:39:23.267085 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f5a726b1-0819-4dd9-bedc-850a106b1cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:23.296600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:23.366315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 08:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:39:23.404686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1d99c-6a8a-4c30-a387-dbbad470a2cb None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:39:23.406357 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-22b1d99c-6a8a-4c30-a387-dbbad470a2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:23.517771 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1d99c-6a8a-4c30-a387-dbbad470a2cb None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:39:23.548671 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:23.596133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3681c43e-f5e4-4088-8147-5fc65630e367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:39:23.596796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681c43e-f5e4-4088-8147-5fc65630e367 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:23.597073 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681c43e-f5e4-4088-8147-5fc65630e367 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:23.597349 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681c43e-f5e4-4088-8147-5fc65630e367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:23.599089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3681c43e-f5e4-4088-8147-5fc65630e367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:23.709578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3681c43e-f5e4-4088-8147-5fc65630e367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:39:23.711056 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3681c43e-f5e4-4088-8147-5fc65630e367 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:39:23.729559 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 729/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:39:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:39:25.854845 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd6c23-1939-477e-a6d3-e98d3bf6c33d None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:25.855397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd6c23-1939-477e-a6d3-e98d3bf6c33d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:25.855517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd6c23-1939-477e-a6d3-e98d3bf6c33d None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:25.856894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 08:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:39:25.862388 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d104687-0fb4-476c-bed2-728a9c31ae6c None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:25.862853 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d104687-0fb4-476c-bed2-728a9c31ae6c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:25.862960 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d104687-0fb4-476c-bed2-728a9c31ae6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:25.929529 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-5d104687-0fb4-476c-bed2-728a9c31ae6c None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:39:25.930738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5d104687-0fb4-476c-bed2-728a9c31ae6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:25.939192 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:26.031707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 08:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:39:26.069779 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd6c23-1939-477e-a6d3-e98d3bf6c33d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:39:26.071337 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ccfd6c23-1939-477e-a6d3-e98d3bf6c33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:26.181668 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd6c23-1939-477e-a6d3-e98d3bf6c33d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:39:26.240374 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:26.262139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58d3cf-214f-460f-9c0e-13ff13f19a5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:39:26.262802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58d3cf-214f-460f-9c0e-13ff13f19a5c service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:26.263087 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58d3cf-214f-460f-9c0e-13ff13f19a5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:26.263363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58d3cf-214f-460f-9c0e-13ff13f19a5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:26.265105 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6a58d3cf-214f-460f-9c0e-13ff13f19a5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:26.385266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58d3cf-214f-460f-9c0e-13ff13f19a5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:39:26.386724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58d3cf-214f-460f-9c0e-13ff13f19a5c service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:39:26.405934 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 731/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:39:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:39:28.478573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f61c7a-5b8f-4a86-9198-e83740a45d9a None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:28.479193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f61c7a-5b8f-4a86-9198-e83740a45d9a None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:28.479193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f61c7a-5b8f-4a86-9198-e83740a45d9a None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:28.480520 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 08:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:39:28.485877 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb00fb4-cd06-4071-bdfc-cc483b97b5ac None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:28.486317 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb00fb4-cd06-4071-bdfc-cc483b97b5ac None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:28.486434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb00fb4-cd06-4071-bdfc-cc483b97b5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:28.554156 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-edb00fb4-cd06-4071-bdfc-cc483b97b5ac None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:39:28.555436 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-edb00fb4-cd06-4071-bdfc-cc483b97b5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:28.595188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:28.654752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 08:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:39:28.693030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f61c7a-5b8f-4a86-9198-e83740a45d9a None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:39:28.695075 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-37f61c7a-5b8f-4a86-9198-e83740a45d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:28.800376 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f61c7a-5b8f-4a86-9198-e83740a45d9a None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:39:28.838735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:28.877010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a2925-66c5-4b49-911a-0adcc88ed20f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:39:28.877662 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a2925-66c5-4b49-911a-0adcc88ed20f service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:28.877946 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a2925-66c5-4b49-911a-0adcc88ed20f service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:28.878227 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a2925-66c5-4b49-911a-0adcc88ed20f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:28.879996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-cb0a2925-66c5-4b49-911a-0adcc88ed20f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:28.995719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a2925-66c5-4b49-911a-0adcc88ed20f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:39:28.997160 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a2925-66c5-4b49-911a-0adcc88ed20f service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:39:29.016539 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 732/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:39:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:39:31.115156 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a26bd-095c-4539-b76a-79015a8ccc57 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:31.116351 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a26bd-095c-4539-b76a-79015a8ccc57 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:31.116351 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a26bd-095c-4539-b76a-79015a8ccc57 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:31.117647 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 08:39:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:39:31.123430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac4090-3e03-4423-8bef-b2ccaa7ebde6 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:31.129274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac4090-3e03-4423-8bef-b2ccaa7ebde6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:31.129274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac4090-3e03-4423-8bef-b2ccaa7ebde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:31.192658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-5aac4090-3e03-4423-8bef-b2ccaa7ebde6 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:39:31.194761 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5aac4090-3e03-4423-8bef-b2ccaa7ebde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:31.214289 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:31.310691 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 08:39:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:39:31.348384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a26bd-095c-4539-b76a-79015a8ccc57 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:39:31.350147 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1e9a26bd-095c-4539-b76a-79015a8ccc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:31.463876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a26bd-095c-4539-b76a-79015a8ccc57 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:39:31.540714 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:31.547399 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7865a19-566c-44cc-9c2f-477afabfe8d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:39:31.548096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7865a19-566c-44cc-9c2f-477afabfe8d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:31.548371 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7865a19-566c-44cc-9c2f-477afabfe8d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:31.548639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7865a19-566c-44cc-9c2f-477afabfe8d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:31.550344 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c7865a19-566c-44cc-9c2f-477afabfe8d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:31.668993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7865a19-566c-44cc-9c2f-477afabfe8d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:39:31.670437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7865a19-566c-44cc-9c2f-477afabfe8d7 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:39:31.689181 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 734/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:39:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:39:33.757505 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5905a-ba39-41f8-8145-3dc8f9455e96 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:33.758356 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5905a-ba39-41f8-8145-3dc8f9455e96 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:33.758519 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5905a-ba39-41f8-8145-3dc8f9455e96 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:33.760363 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 08:39:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:39:33.765768 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c0cd2-c748-4569-a7a8-e03eb2218bec None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:33.766221 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c0cd2-c748-4569-a7a8-e03eb2218bec None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:33.766338 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c0cd2-c748-4569-a7a8-e03eb2218bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:33.833442 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-498c0cd2-c748-4569-a7a8-e03eb2218bec None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:39:33.834728 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-498c0cd2-c748-4569-a7a8-e03eb2218bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:33.886618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:33.936085 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 08:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:39:33.973353 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5905a-ba39-41f8-8145-3dc8f9455e96 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:39:33.975656 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c3e5905a-ba39-41f8-8145-3dc8f9455e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:34.079249 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5905a-ba39-41f8-8145-3dc8f9455e96 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:39:34.128894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:34.156670 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39829d-4af6-453a-9771-f1e3d9669a0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:39:34.157332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39829d-4af6-453a-9771-f1e3d9669a0a service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:34.157615 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39829d-4af6-453a-9771-f1e3d9669a0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:34.157890 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39829d-4af6-453a-9771-f1e3d9669a0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:34.159600 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1b39829d-4af6-453a-9771-f1e3d9669a0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:34.275370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39829d-4af6-453a-9771-f1e3d9669a0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:39:34.276825 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39829d-4af6-453a-9771-f1e3d9669a0a service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:39:34.295727 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 735/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:39:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:39:43.934190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=22 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:44.293360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=66 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:54.390895 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291abfe-d62f-442e-a737-56f2fee26cd6 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:54.391446 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291abfe-d62f-442e-a737-56f2fee26cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:54.391446 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291abfe-d62f-442e-a737-56f2fee26cd6 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:54.392777 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 736/1471] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:39:54.398195 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f8cf6-cfa0-4321-9c1b-36073f91876b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:54.398644 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f8cf6-cfa0-4321-9c1b-36073f91876b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:54.398749 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f8cf6-cfa0-4321-9c1b-36073f91876b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:54.469348 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-e09f8cf6-cfa0-4321-9c1b-36073f91876b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:39:54.470583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e09f8cf6-cfa0-4321-9c1b-36073f91876b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:54.578646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 736/1472] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:39:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:39:54.628011 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d255c-4e8e-490b-96b3-be5d7d25ce8b None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:54.628303 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d255c-4e8e-490b-96b3-be5d7d25ce8b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:54.628582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d255c-4e8e-490b-96b3-be5d7d25ce8b None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:54.630606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 737/1473] 10.222.0.57 () {54 vars in 961 bytes} [Mon Jul 18 08:39:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:39:54.636056 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abce114-00d5-4fbe-979f-f1f8567fb2d1 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:54.636348 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abce114-00d5-4fbe-979f-f1f8567fb2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:54.636625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abce114-00d5-4fbe-979f-f1f8567fb2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:54.712152 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-4abce114-00d5-4fbe-979f-f1f8567fb2d1 None None] MFA Rules not processed for user `694088ed30b94bad9d3b16f3766252df`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:39:54.713373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4abce114-00d5-4fbe-979f-f1f8567fb2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:54.735456 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:54.792666 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 737/1474] 10.222.0.57 () {58 vars in 1058 bytes} [Mon Jul 18 08:39:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:39:54.800294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d255c-4e8e-490b-96b3-be5d7d25ce8b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:39:54.802492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8d6d255c-4e8e-490b-96b3-be5d7d25ce8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:54.908624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d255c-4e8e-490b-96b3-be5d7d25ce8b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:39:54.985761 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52e92d-d300-4b2d-b26d-76551d28f0f7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '694088ed30b94bad9d3b16f3766252df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:39:54.986323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52e92d-d300-4b2d-b26d-76551d28f0f7 service glance] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:54.986609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52e92d-d300-4b2d-b26d-76551d28f0f7 service glance] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:54.986883 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52e92d-d300-4b2d-b26d-76551d28f0f7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:54.988576 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ad52e92d-d300-4b2d-b26d-76551d28f0f7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:55.043876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:55.099430 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52e92d-d300-4b2d-b26d-76551d28f0f7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:39:55.100918 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52e92d-d300-4b2d-b26d-76551d28f0f7 service glance] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:39:55.120444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 738/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:39:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:39:57.289529 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061840c8-4ff9-4cef-ae80-bae47846d09d None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:57.290104 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061840c8-4ff9-4cef-ae80-bae47846d09d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:57.290104 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061840c8-4ff9-4cef-ae80-bae47846d09d None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:57.291510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 738/1476] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:39:57.297092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e306c3d-60b8-494c-a17c-2b661491388a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:57.297526 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e306c3d-60b8-494c-a17c-2b661491388a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:57.297641 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e306c3d-60b8-494c-a17c-2b661491388a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:57.362922 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8e306c3d-60b8-494c-a17c-2b661491388a None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:39:57.364687 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8e306c3d-60b8-494c-a17c-2b661491388a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:57.462298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 739/1477] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:39:57.484925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061840c8-4ff9-4cef-ae80-bae47846d09d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:39:57.486435 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-061840c8-4ff9-4cef-ae80-bae47846d09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:57.616885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061840c8-4ff9-4cef-ae80-bae47846d09d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:39:57.669168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:39:57.715310 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61f224-182e-4d5a-9ff2-7f9ecc80380b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:39:57.716007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61f224-182e-4d5a-9ff2-7f9ecc80380b admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:57.716292 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61f224-182e-4d5a-9ff2-7f9ecc80380b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:57.716820 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61f224-182e-4d5a-9ff2-7f9ecc80380b admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:57.717569 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61f224-182e-4d5a-9ff2-7f9ecc80380b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:39:57.719360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: WARNING py.warnings [None req-ed61f224-182e-4d5a-9ff2-7f9ecc80380b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 08:39:57.719360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: warnings.warn(msg) Jul 18 08:39:57.719360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: Jul 18 08:39:57.720556 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61f224-182e-4d5a-9ff2-7f9ecc80380b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:39:57.741132 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 739/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Jul 18 08:39:57 2022] GET /identity/v3/projects => generated 2553 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:39:59.806067 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648eca2-afab-4cde-ac70-03fd66d028f8 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:59.806603 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648eca2-afab-4cde-ac70-03fd66d028f8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:59.806603 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648eca2-afab-4cde-ac70-03fd66d028f8 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:59.807943 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 740/1479] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:39:59.813386 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc063e-cd52-4546-8d3f-b687a6afa9d2 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:39:59.813831 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc063e-cd52-4546-8d3f-b687a6afa9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:39:59.813944 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc063e-cd52-4546-8d3f-b687a6afa9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:39:59.886010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-4fcc063e-cd52-4546-8d3f-b687a6afa9d2 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:39:59.887220 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4fcc063e-cd52-4546-8d3f-b687a6afa9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:39:59.985725 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 740/1480] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:40:00.029343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648eca2-afab-4cde-ac70-03fd66d028f8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:40:00.030884 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5648eca2-afab-4cde-ac70-03fd66d028f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:40:00.041079 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:40:00.181025 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648eca2-afab-4cde-ac70-03fd66d028f8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:40:00.264448 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:40:00.298731 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1658aab-93f0-4e95-a66f-a8ec6ccd68b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:40:00.299428 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1658aab-93f0-4e95-a66f-a8ec6ccd68b3 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:40:00.299819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1658aab-93f0-4e95-a66f-a8ec6ccd68b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:40:00.300488 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1658aab-93f0-4e95-a66f-a8ec6ccd68b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:40:00.303194 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c1658aab-93f0-4e95-a66f-a8ec6ccd68b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:40:00.395191 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1658aab-93f0-4e95-a66f-a8ec6ccd68b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:40:00.396882 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1658aab-93f0-4e95-a66f-a8ec6ccd68b3 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:40:00.415788 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 741/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:40:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:40:02.702701 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750dd82-0a43-4dc4-90e8-4c48526ceaad None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:40:02.703272 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750dd82-0a43-4dc4-90e8-4c48526ceaad None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:40:02.703272 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750dd82-0a43-4dc4-90e8-4c48526ceaad None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:40:02.704688 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 741/1482] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:40:02.710575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be295781-2eb5-47ba-891d-9cdcee544524 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:40:02.711046 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be295781-2eb5-47ba-891d-9cdcee544524 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:40:02.711151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be295781-2eb5-47ba-891d-9cdcee544524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:40:02.780319 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-be295781-2eb5-47ba-891d-9cdcee544524 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:40:02.781538 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-be295781-2eb5-47ba-891d-9cdcee544524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:40:02.829229 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:40:02.882019 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 742/1483] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:40:02.918342 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750dd82-0a43-4dc4-90e8-4c48526ceaad None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:40:02.919952 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a750dd82-0a43-4dc4-90e8-4c48526ceaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:40:02.943890 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:40:03.026152 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750dd82-0a43-4dc4-90e8-4c48526ceaad None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:40:03.100168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9a627-0643-4c86-b2cc-edc8eae95933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:40:03.100819 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9a627-0643-4c86-b2cc-edc8eae95933 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:40:03.101100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9a627-0643-4c86-b2cc-edc8eae95933 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:40:03.101373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9a627-0643-4c86-b2cc-edc8eae95933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:40:03.103025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-32b9a627-0643-4c86-b2cc-edc8eae95933 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:40:03.178424 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:40:03.212866 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9a627-0643-4c86-b2cc-edc8eae95933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:40:03.214296 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9a627-0643-4c86-b2cc-edc8eae95933 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:40:03.233766 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 742/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:40:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:40:05.320477 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d047fc-4b70-40bf-94fb-d1f25ec7a54f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:40:05.321049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d047fc-4b70-40bf-94fb-d1f25ec7a54f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:40:05.321049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d047fc-4b70-40bf-94fb-d1f25ec7a54f None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:40:05.322428 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 743/1485] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:40:05.327922 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3c1ed-9a1d-422c-a249-9a0fd849c9bf None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:40:05.328359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3c1ed-9a1d-422c-a249-9a0fd849c9bf None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:40:05.328486 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3c1ed-9a1d-422c-a249-9a0fd849c9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:40:05.393448 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-84e3c1ed-9a1d-422c-a249-9a0fd849c9bf None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:40:05.395315 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-84e3c1ed-9a1d-422c-a249-9a0fd849c9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:40:05.492686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 743/1486] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:40:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:40:05.531231 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d047fc-4b70-40bf-94fb-d1f25ec7a54f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:40:05.532836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-16d047fc-4b70-40bf-94fb-d1f25ec7a54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:40:05.641331 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d047fc-4b70-40bf-94fb-d1f25ec7a54f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:40:05.720202 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:40:05.722040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce899227-e3b5-48f9-8d65-e05899d8352d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:40:05.722720 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce899227-e3b5-48f9-8d65-e05899d8352d service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:40:05.722986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce899227-e3b5-48f9-8d65-e05899d8352d service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:40:05.724074 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce899227-e3b5-48f9-8d65-e05899d8352d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:40:05.725906 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ce899227-e3b5-48f9-8d65-e05899d8352d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:40:05.840664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce899227-e3b5-48f9-8d65-e05899d8352d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:40:05.842390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce899227-e3b5-48f9-8d65-e05899d8352d service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:40:05.860927 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 744/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:40:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:40:08.097073 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69189d-5f3d-41e0-bf2f-b0922af1a038 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:40:08.098370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69189d-5f3d-41e0-bf2f-b0922af1a038 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:40:08.098370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69189d-5f3d-41e0-bf2f-b0922af1a038 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:40:08.099681 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 744/1488] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:40:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:40:08.105234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d2c5d-e854-4445-beed-0ce74bbf9c07 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:40:08.105658 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d2c5d-e854-4445-beed-0ce74bbf9c07 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:40:08.105928 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d2c5d-e854-4445-beed-0ce74bbf9c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:40:08.176935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-2b1d2c5d-e854-4445-beed-0ce74bbf9c07 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:40:08.178171 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2b1d2c5d-e854-4445-beed-0ce74bbf9c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:40:08.236483 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:40:08.285979 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 745/1489] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:40:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:40:08.328258 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69189d-5f3d-41e0-bf2f-b0922af1a038 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:40:08.329798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3f69189d-5f3d-41e0-bf2f-b0922af1a038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:40:08.366666 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:40:08.436240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69189d-5f3d-41e0-bf2f-b0922af1a038 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:40:08.511339 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1bbcbb-f88d-466c-ae90-8a6c10056d15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:40:08.512026 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bbcbb-f88d-466c-ae90-8a6c10056d15 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:40:08.512313 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bbcbb-f88d-466c-ae90-8a6c10056d15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:40:08.512598 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bbcbb-f88d-466c-ae90-8a6c10056d15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:40:08.514263 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-df1bbcbb-f88d-466c-ae90-8a6c10056d15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:40:08.598228 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:40:08.627718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1bbcbb-f88d-466c-ae90-8a6c10056d15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:40:08.630706 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1bbcbb-f88d-466c-ae90-8a6c10056d15 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:40:08.649280 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 745/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:40:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:40:11.099886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84cebe-45a2-47f1-8c6d-25eab49ec0b7 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:40:11.100567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84cebe-45a2-47f1-8c6d-25eab49ec0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:40:11.100567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84cebe-45a2-47f1-8c6d-25eab49ec0b7 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:40:11.101782 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 746/1491] 10.222.0.57 () {54 vars in 926 bytes} [Mon Jul 18 08:40:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:40:11.107366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3db0e-14fd-4ca9-8b36-6901e7afb133 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:40:11.112535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3db0e-14fd-4ca9-8b36-6901e7afb133 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:40:11.112535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3db0e-14fd-4ca9-8b36-6901e7afb133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:40:11.176272 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-77b3db0e-14fd-4ca9-8b36-6901e7afb133 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:40:11.185715 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-77b3db0e-14fd-4ca9-8b36-6901e7afb133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:40:11.278721 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 746/1492] 10.222.0.57 () {58 vars in 1023 bytes} [Mon Jul 18 08:40:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:40:11.318755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84cebe-45a2-47f1-8c6d-25eab49ec0b7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:40:11.320343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4b84cebe-45a2-47f1-8c6d-25eab49ec0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:40:11.427622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84cebe-45a2-47f1-8c6d-25eab49ec0b7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:40:11.501739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91edd1c-a87a-49c0-9a3a-3fb98e5c02e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:40:11.502387 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91edd1c-a87a-49c0-9a3a-3fb98e5c02e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:40:11.502671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91edd1c-a87a-49c0-9a3a-3fb98e5c02e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:40:11.502948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91edd1c-a87a-49c0-9a3a-3fb98e5c02e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:40:11.505584 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f91edd1c-a87a-49c0-9a3a-3fb98e5c02e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:40:11.514975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:40:11.619800 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91edd1c-a87a-49c0-9a3a-3fb98e5c02e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:40:11.621244 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91edd1c-a87a-49c0-9a3a-3fb98e5c02e8 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:40:11.640100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 747/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:40:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:40:21.276811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=82 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:40:21.637753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=50 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:42:14.672474 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c9160-7480-41ae-b6f9-501c690aeecb None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:42:14.673068 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c9160-7480-41ae-b6f9-501c690aeecb None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:42:14.673068 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c9160-7480-41ae-b6f9-501c690aeecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:42:14.758529 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-451c9160-7480-41ae-b6f9-501c690aeecb None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:42:14.759774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-451c9160-7480-41ae-b6f9-501c690aeecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:42:14.865513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 747/1494] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:42:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:42:14.873741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe93bf1-9e8d-4fa9-b37c-9f2e7e9aff47 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:42:14.874278 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe93bf1-9e8d-4fa9-b37c-9f2e7e9aff47 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:42:14.874381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe93bf1-9e8d-4fa9-b37c-9f2e7e9aff47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:42:14.895235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-cfe93bf1-9e8d-4fa9-b37c-9f2e7e9aff47 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:42:14.896498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cfe93bf1-9e8d-4fa9-b37c-9f2e7e9aff47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:42:14.905179 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 748/1495] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:42:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:42:14.910063 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c9160-7480-41ae-b6f9-501c690aeecb None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:42:14.920524 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c9160-7480-41ae-b6f9-501c690aeecb None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:42:14.924923 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683b61c-af1a-4c5e-9bb9-367e8e952e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:42:14.926109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683b61c-af1a-4c5e-9bb9-367e8e952e5b admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:42:14.926502 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683b61c-af1a-4c5e-9bb9-367e8e952e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:42:14.926784 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683b61c-af1a-4c5e-9bb9-367e8e952e5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:42:14.927508 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683b61c-af1a-4c5e-9bb9-367e8e952e5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:42:14.927848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683b61c-af1a-4c5e-9bb9-367e8e952e5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:42:14.928865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: WARNING py.warnings [None req-a683b61c-af1a-4c5e-9bb9-367e8e952e5b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 08:42:14.928865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: warnings.warn(msg) Jul 18 08:42:14.928865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: Jul 18 08:42:14.929744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683b61c-af1a-4c5e-9bb9-367e8e952e5b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:42:14.943274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 748/1496] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:42:14.952025 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe93bf1-9e8d-4fa9-b37c-9f2e7e9aff47 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:42:14.962272 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe93bf1-9e8d-4fa9-b37c-9f2e7e9aff47 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:42:14.966978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266579e-b5d7-404e-a4ea-2ddd52b5fef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:42:14.967646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266579e-b5d7-404e-a4ea-2ddd52b5fef2 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:42:14.967965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266579e-b5d7-404e-a4ea-2ddd52b5fef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:42:14.968246 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266579e-b5d7-404e-a4ea-2ddd52b5fef2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:42:14.968954 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266579e-b5d7-404e-a4ea-2ddd52b5fef2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:42:14.970785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-5266579e-b5d7-404e-a4ea-2ddd52b5fef2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 08:42:14.970785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Jul 18 08:42:14.970785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: Jul 18 08:42:14.972044 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266579e-b5d7-404e-a4ea-2ddd52b5fef2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:42:15.100954 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 749/1497] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:42:14 2022] POST /identity/v3/projects => generated 379 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:42:15.105319 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683b61c-af1a-4c5e-9bb9-367e8e952e5b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:42:15.116440 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683b61c-af1a-4c5e-9bb9-367e8e952e5b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:42:15.175027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:42:15.182865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0b8a6-26d1-4043-a69d-4c88051598d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:42:15.186288 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0b8a6-26d1-4043-a69d-4c88051598d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:42:15.186288 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0b8a6-26d1-4043-a69d-4c88051598d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:42:15.186288 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0b8a6-26d1-4043-a69d-4c88051598d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:42:15.186288 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0b8a6-26d1-4043-a69d-4c88051598d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:42:15.186288 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: WARNING py.warnings [None req-f0f0b8a6-26d1-4043-a69d-4c88051598d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 08:42:15.186288 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: warnings.warn(msg) Jul 18 08:42:15.186288 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: Jul 18 08:42:15.187353 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0b8a6-26d1-4043-a69d-4c88051598d5 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:42:15.264148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 749/1498] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:42:15 2022] POST /identity/v3/users => generated 290 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:42:15.268604 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266579e-b5d7-404e-a4ea-2ddd52b5fef2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:42:15.280006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266579e-b5d7-404e-a4ea-2ddd52b5fef2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:42:15.285504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2434590-7f35-4cd2-aaba-65b4a8d84573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:42:15.286956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2434590-7f35-4cd2-aaba-65b4a8d84573 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:42:15.287357 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2434590-7f35-4cd2-aaba-65b4a8d84573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:42:15.287639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2434590-7f35-4cd2-aaba-65b4a8d84573 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:42:15.288425 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2434590-7f35-4cd2-aaba-65b4a8d84573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:42:15.289450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-d2434590-7f35-4cd2-aaba-65b4a8d84573 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 08:42:15.289450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Jul 18 08:42:15.289450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: Jul 18 08:42:15.290358 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2434590-7f35-4cd2-aaba-65b4a8d84573 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:42:15.305685 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 750/1499] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:42:15.309767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0b8a6-26d1-4043-a69d-4c88051598d5 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:42:15.321004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0b8a6-26d1-4043-a69d-4c88051598d5 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:42:15.325834 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110cc4b-59bf-4863-9e6b-0134d90b3494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:42:15.326507 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110cc4b-59bf-4863-9e6b-0134d90b3494 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:42:15.326796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110cc4b-59bf-4863-9e6b-0134d90b3494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:42:15.327113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110cc4b-59bf-4863-9e6b-0134d90b3494 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:42:15.327904 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110cc4b-59bf-4863-9e6b-0134d90b3494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:42:15.328913 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: WARNING py.warnings [None req-a110cc4b-59bf-4863-9e6b-0134d90b3494 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 08:42:15.328913 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: warnings.warn(msg) Jul 18 08:42:15.328913 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: Jul 18 08:42:15.329807 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110cc4b-59bf-4863-9e6b-0134d90b3494 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:42:15.343944 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 750/1500] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:42:15.349391 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2434590-7f35-4cd2-aaba-65b4a8d84573 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:42:15.359361 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2434590-7f35-4cd2-aaba-65b4a8d84573 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:42:15.364803 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70353ee-3800-4921-bffb-0161ab080bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:42:15.365779 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70353ee-3800-4921-bffb-0161ab080bad admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:42:15.366060 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70353ee-3800-4921-bffb-0161ab080bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:42:15.366330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70353ee-3800-4921-bffb-0161ab080bad admin admin] PATH_INFO: `/v3/projects/f0d0a8b3f689424eb1cf9131f61a11ca/users/6df3591e751f42d48f0751011cfe4b80/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:42:15.383410 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70353ee-3800-4921-bffb-0161ab080bad admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0d0a8b3f689424eb1cf9131f61a11ca, user_id=6df3591e751f42d48f0751011cfe4b80, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:42:15.385392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-a70353ee-3800-4921-bffb-0161ab080bad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 08:42:15.385392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Jul 18 08:42:15.385392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: Jul 18 08:42:15.388699 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70353ee-3800-4921-bffb-0161ab080bad admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:42:15.462768 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 751/1501] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:42:15 2022] PUT /identity/v3/projects/f0d0a8b3f689424eb1cf9131f61a11ca/users/6df3591e751f42d48f0751011cfe4b80/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:42:15.476455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdf726-eba3-43a4-a851-df830c06a91d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:42:15.476833 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdf726-eba3-43a4-a851-df830c06a91d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:42:15.477006 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdf726-eba3-43a4-a851-df830c06a91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:42:15.504230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-efcdf726-eba3-43a4-a851-df830c06a91d None None] MFA Rules not processed for user `6df3591e751f42d48f0751011cfe4b80`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:42:15.505525 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-efcdf726-eba3-43a4-a851-df830c06a91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:42:15.571099 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 751/1502] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:42:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:42:15.585481 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd725e89-2782-4357-8f15-966e619634c2 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:42:15.585773 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd725e89-2782-4357-8f15-966e619634c2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:42:15.586052 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd725e89-2782-4357-8f15-966e619634c2 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:42:15.587704 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 752/1503] 10.222.0.57 () {54 vars in 964 bytes} [Mon Jul 18 08:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:42:15.592940 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de6fe5-69ee-435b-80aa-8bfc7a05ddb6 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:42:15.593308 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de6fe5-69ee-435b-80aa-8bfc7a05ddb6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:42:15.593632 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de6fe5-69ee-435b-80aa-8bfc7a05ddb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:42:15.670460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-89de6fe5-69ee-435b-80aa-8bfc7a05ddb6 None None] MFA Rules not processed for user `d5f9a129021445ae8154a5e6941291e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:42:15.671753 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-89de6fe5-69ee-435b-80aa-8bfc7a05ddb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:42:15.752354 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 752/1504] 10.222.0.57 () {58 vars in 1061 bytes} [Mon Jul 18 08:42:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:42:15.760188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd725e89-2782-4357-8f15-966e619634c2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:42:15.771704 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd725e89-2782-4357-8f15-966e619634c2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:42:15.779450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7904d-1afd-44b3-82d8-e12ba7b6b933 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:42:15.780142 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7904d-1afd-44b3-82d8-e12ba7b6b933 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:42:15.780429 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7904d-1afd-44b3-82d8-e12ba7b6b933 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:42:15.780705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7904d-1afd-44b3-82d8-e12ba7b6b933 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:42:15.791126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7904d-1afd-44b3-82d8-e12ba7b6b933 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:42:15.792729 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7904d-1afd-44b3-82d8-e12ba7b6b933 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:42:15.802643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 753/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:42:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:42:15.846402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c5fe2-9a50-4bc8-b404-8ded59395d90 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:42:15.846690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c5fe2-9a50-4bc8-b404-8ded59395d90 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:42:15.846968 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c5fe2-9a50-4bc8-b404-8ded59395d90 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:42:15.848878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 753/1506] 10.222.0.57 () {52 vars in 829 bytes} [Mon Jul 18 08:42:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:42:15.853551 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7904d-1afd-44b3-82d8-e12ba7b6b933 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:42:15.916403 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7904d-1afd-44b3-82d8-e12ba7b6b933 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:42:15.921166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7600d7-962c-46e1-b257-d9a8b087beb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:42:15.922566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7600d7-962c-46e1-b257-d9a8b087beb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:42:15.922566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7600d7-962c-46e1-b257-d9a8b087beb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:42:15.922899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7600d7-962c-46e1-b257-d9a8b087beb4 admin admin] PATH_INFO: `/v3/users/6df3591e751f42d48f0751011cfe4b80` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:42:15.923897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7600d7-962c-46e1-b257-d9a8b087beb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6df3591e751f42d48f0751011cfe4b80)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:42:15.924916 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-1d7600d7-962c-46e1-b257-d9a8b087beb4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 08:42:15.924916 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Jul 18 08:42:15.924916 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: Jul 18 08:42:15.927043 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7600d7-962c-46e1-b257-d9a8b087beb4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:42:16.140445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1d7600d7-962c-46e1-b257-d9a8b087beb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6df3591e751f42d48f0751011cfe4b80'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:42:16.141518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1d7600d7-962c-46e1-b257-d9a8b087beb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6df3591e751f42d48f0751011cfe4b80'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:42:16.142017 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1d7600d7-962c-46e1-b257-d9a8b087beb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6df3591e751f42d48f0751011cfe4b80'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:42:16.163641 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1d7600d7-962c-46e1-b257-d9a8b087beb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6df3591e751f42d48f0751011cfe4b80'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:42:16.179445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1d7600d7-962c-46e1-b257-d9a8b087beb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6df3591e751f42d48f0751011cfe4b80'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:42:16.183999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:42:16.338526 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 754/1507] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:42:15 2022] DELETE /identity/v3/users/6df3591e751f42d48f0751011cfe4b80 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:42:16.342924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198eeb3-dc6f-4c1a-a7c7-baed83b19e9f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:42:16.343208 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198eeb3-dc6f-4c1a-a7c7-baed83b19e9f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:42:16.343483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198eeb3-dc6f-4c1a-a7c7-baed83b19e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:42:16.363269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2198eeb3-dc6f-4c1a-a7c7-baed83b19e9f None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:42:16.364631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2198eeb3-dc6f-4c1a-a7c7-baed83b19e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:42:16.374068 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:42:16.426978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 754/1508] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:42:16.436777 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7600d7-962c-46e1-b257-d9a8b087beb4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:42:16.438192 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1d7600d7-962c-46e1-b257-d9a8b087beb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:42:16.521480 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7600d7-962c-46e1-b257-d9a8b087beb4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:42:16.525374 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbbd87-c5a4-49de-aa56-1d6fa7af644c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:42:16.526032 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbbd87-c5a4-49de-aa56-1d6fa7af644c service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:42:16.526315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbbd87-c5a4-49de-aa56-1d6fa7af644c service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:42:16.526591 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbbd87-c5a4-49de-aa56-1d6fa7af644c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:42:16.537185 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbbd87-c5a4-49de-aa56-1d6fa7af644c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:42:16.538616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbbd87-c5a4-49de-aa56-1d6fa7af644c service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:42:16.548280 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 755/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:42:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:42:17.065405 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198eeb3-dc6f-4c1a-a7c7-baed83b19e9f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:42:17.069863 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2198eeb3-dc6f-4c1a-a7c7-baed83b19e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:42:17.080301 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198eeb3-dc6f-4c1a-a7c7-baed83b19e9f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:42:17.085461 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696b232-b93c-4aa3-8129-f29a8b6d2e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:42:17.086592 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696b232-b93c-4aa3-8129-f29a8b6d2e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:42:17.086891 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696b232-b93c-4aa3-8129-f29a8b6d2e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:42:17.087223 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696b232-b93c-4aa3-8129-f29a8b6d2e4a admin admin] PATH_INFO: `/v3/projects/f0d0a8b3f689424eb1cf9131f61a11ca` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:42:17.088423 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696b232-b93c-4aa3-8129-f29a8b6d2e4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0d0a8b3f689424eb1cf9131f61a11ca)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:42:17.092753 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: WARNING py.warnings [None req-3696b232-b93c-4aa3-8129-f29a8b6d2e4a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 08:42:17.092753 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: warnings.warn(msg) Jul 18 08:42:17.092753 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: Jul 18 08:42:17.092753 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696b232-b93c-4aa3-8129-f29a8b6d2e4a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:42:17.208706 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3696b232-b93c-4aa3-8129-f29a8b6d2e4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0d0a8b3f689424eb1cf9131f61a11ca', 'request_id': 'req-3696b232-b93c-4aa3-8129-f29a8b6d2e4a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:42:17.215146 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3696b232-b93c-4aa3-8129-f29a8b6d2e4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0d0a8b3f689424eb1cf9131f61a11ca', 'request_id': 'req-3696b232-b93c-4aa3-8129-f29a8b6d2e4a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:42:17.228523 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3696b232-b93c-4aa3-8129-f29a8b6d2e4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0d0a8b3f689424eb1cf9131f61a11ca', 'request_id': 'req-3696b232-b93c-4aa3-8129-f29a8b6d2e4a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:42:17.304178 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3696b232-b93c-4aa3-8129-f29a8b6d2e4a admin admin] The token cache is being invalidate because project f0d0a8b3f689424eb1cf9131f61a11ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:42:17.304468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3696b232-b93c-4aa3-8129-f29a8b6d2e4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:42:17.305129 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3696b232-b93c-4aa3-8129-f29a8b6d2e4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:42:17.306828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 755/1510] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:42:17 2022] DELETE /identity/v3/projects/f0d0a8b3f689424eb1cf9131f61a11ca => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:42:26.543912 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:42:27.299347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:43:56.479955 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e97948-26a1-4ae5-a680-1ec1cc7e7f24 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:43:56.480559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e97948-26a1-4ae5-a680-1ec1cc7e7f24 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:43:56.480559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e97948-26a1-4ae5-a680-1ec1cc7e7f24 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:43:56.481923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 08:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:43:56.487475 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d8b38-ab31-4c29-993b-d25475820258 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:43:56.487985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d8b38-ab31-4c29-993b-d25475820258 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:43:56.488084 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d8b38-ab31-4c29-993b-d25475820258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:43:56.507004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-148d8b38-ab31-4c29-993b-d25475820258 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:43:56.508220 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-148d8b38-ab31-4c29-993b-d25475820258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:43:56.570251 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 08:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:43:56.726904 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e97948-26a1-4ae5-a680-1ec1cc7e7f24 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:43:56.728647 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-93e97948-26a1-4ae5-a680-1ec1cc7e7f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:43:56.829045 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e97948-26a1-4ae5-a680-1ec1cc7e7f24 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:43:56.834780 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc76b2-d144-4cde-9a65-7f0a1676b188 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:43:56.835435 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc76b2-d144-4cde-9a65-7f0a1676b188 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:43:56.835745 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc76b2-d144-4cde-9a65-7f0a1676b188 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:43:56.836024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc76b2-d144-4cde-9a65-7f0a1676b188 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:43:56.844930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc76b2-d144-4cde-9a65-7f0a1676b188 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:43:56.846731 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc76b2-d144-4cde-9a65-7f0a1676b188 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:43:56.856238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 757/1513] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:43:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:43:57.695235 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38534780-addc-43a3-935a-a423674eac76 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:43:57.695812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38534780-addc-43a3-935a-a423674eac76 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:43:57.695921 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38534780-addc-43a3-935a-a423674eac76 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:43:57.697266 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 08:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:43:57.702065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3f546-13f0-4a18-8d03-9962137fa57b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:43:57.702515 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3f546-13f0-4a18-8d03-9962137fa57b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:43:57.702650 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3f546-13f0-4a18-8d03-9962137fa57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:43:57.722628 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f2d3f546-13f0-4a18-8d03-9962137fa57b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:43:57.724995 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f2d3f546-13f0-4a18-8d03-9962137fa57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:43:57.734203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 08:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:43:57.891069 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38534780-addc-43a3-935a-a423674eac76 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:43:57.905507 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38534780-addc-43a3-935a-a423674eac76 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:43:57.911737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711ee27-03be-4840-833b-29f52b036a93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:43:57.912392 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711ee27-03be-4840-833b-29f52b036a93 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:43:57.912676 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711ee27-03be-4840-833b-29f52b036a93 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:43:57.912955 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711ee27-03be-4840-833b-29f52b036a93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:43:57.921790 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711ee27-03be-4840-833b-29f52b036a93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:43:57.923398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711ee27-03be-4840-833b-29f52b036a93 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:43:57.933562 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 758/1516] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:43:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:43:58.563104 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a1e05-3c4c-4372-a176-6c98cb0ee7f2 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:43:58.563317 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a1e05-3c4c-4372-a176-6c98cb0ee7f2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:43:58.563583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a1e05-3c4c-4372-a176-6c98cb0ee7f2 None None] PATH_INFO: `/v3` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:43:58.564956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 759/1517] 10.222.0.47 () {54 vars in 925 bytes} [Mon Jul 18 08:43:58 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:43:58.571171 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e1a72-3e62-4687-bcb7-094429ab9f82 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:43:58.571462 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e1a72-3e62-4687-bcb7-094429ab9f82 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:43:58.571774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e1a72-3e62-4687-bcb7-094429ab9f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:43:58.620349 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-215e1a72-3e62-4687-bcb7-094429ab9f82 None None] MFA Rules not processed for user `d606fd7ef269460a8110223932f02d19`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:43:58.621534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-215e1a72-3e62-4687-bcb7-094429ab9f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:43:58.693569 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 759/1518] 10.222.0.47 () {58 vars in 1015 bytes} [Mon Jul 18 08:43:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:44:02.723680 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ccf5d-cce5-4cdc-ac53-3e5b9a57ee41 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:02.724239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ccf5d-cce5-4cdc-ac53-3e5b9a57ee41 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:02.724404 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ccf5d-cce5-4cdc-ac53-3e5b9a57ee41 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:02.727213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 760/1519] 10.222.0.57 () {54 vars in 961 bytes} [Mon Jul 18 08:44:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:44:02.735639 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630614a-817b-42f2-b374-93548f741aa8 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:02.736150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630614a-817b-42f2-b374-93548f741aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:02.736269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630614a-817b-42f2-b374-93548f741aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:02.754266 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-8630614a-817b-42f2-b374-93548f741aa8 None None] MFA Rules not processed for user `d606fd7ef269460a8110223932f02d19`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:44:02.755463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8630614a-817b-42f2-b374-93548f741aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:44:02.763980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 760/1520] 10.222.0.57 () {58 vars in 1058 bytes} [Mon Jul 18 08:44:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:44:02.771509 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ccf5d-cce5-4cdc-ac53-3e5b9a57ee41 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:44:02.785434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ccf5d-cce5-4cdc-ac53-3e5b9a57ee41 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:44:02.790478 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bd074-cd7c-4e30-afe0-9edc5636af7f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd606fd7ef269460a8110223932f02d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:44:02.791110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bd074-cd7c-4e30-afe0-9edc5636af7f service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:02.791395 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bd074-cd7c-4e30-afe0-9edc5636af7f service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:02.791690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bd074-cd7c-4e30-afe0-9edc5636af7f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:02.800825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bd074-cd7c-4e30-afe0-9edc5636af7f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:44:02.802407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bd074-cd7c-4e30-afe0-9edc5636af7f service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:44:02.811940 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 761/1521] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:44:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:44:09.192563 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e300c7-a140-4a8d-bf5b-59e7fbbf2728 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:09.193159 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e300c7-a140-4a8d-bf5b-59e7fbbf2728 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:09.193159 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e300c7-a140-4a8d-bf5b-59e7fbbf2728 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:09.194518 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 08:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:44:09.199424 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7111cc-5459-4d0f-bed9-0c3c26dc2da4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:09.199909 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7111cc-5459-4d0f-bed9-0c3c26dc2da4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:09.200039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7111cc-5459-4d0f-bed9-0c3c26dc2da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:09.222052 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-cc7111cc-5459-4d0f-bed9-0c3c26dc2da4 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:44:09.223294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cc7111cc-5459-4d0f-bed9-0c3c26dc2da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:44:09.231305 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 08:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:44:09.386939 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e300c7-a140-4a8d-bf5b-59e7fbbf2728 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:44:09.401891 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e300c7-a140-4a8d-bf5b-59e7fbbf2728 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:44:09.407933 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a04ee-52bc-4565-94fc-0096afbfca05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:44:09.408583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a04ee-52bc-4565-94fc-0096afbfca05 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:09.408869 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a04ee-52bc-4565-94fc-0096afbfca05 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:09.409136 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a04ee-52bc-4565-94fc-0096afbfca05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:09.417828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a04ee-52bc-4565-94fc-0096afbfca05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:44:09.419360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a04ee-52bc-4565-94fc-0096afbfca05 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:44:09.430186 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 762/1524] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:44:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:44:10.265494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92720755-7864-47bf-b394-979fc883e498 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:10.266102 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92720755-7864-47bf-b394-979fc883e498 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:10.266102 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92720755-7864-47bf-b394-979fc883e498 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:10.267420 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 08:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:44:10.272369 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca168ae2-eea2-42fe-96d7-205a5bb0e132 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:10.272801 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca168ae2-eea2-42fe-96d7-205a5bb0e132 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:10.272944 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca168ae2-eea2-42fe-96d7-205a5bb0e132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:10.291650 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-ca168ae2-eea2-42fe-96d7-205a5bb0e132 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:44:10.292878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ca168ae2-eea2-42fe-96d7-205a5bb0e132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:44:10.301500 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 08:44:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:44:10.452992 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92720755-7864-47bf-b394-979fc883e498 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:44:10.467566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92720755-7864-47bf-b394-979fc883e498 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:44:10.473544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5840b-a815-4adf-b818-733aa26ce645 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:44:10.474187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5840b-a815-4adf-b818-733aa26ce645 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:10.474474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5840b-a815-4adf-b818-733aa26ce645 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:10.474757 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5840b-a815-4adf-b818-733aa26ce645 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:10.483030 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5840b-a815-4adf-b818-733aa26ce645 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:44:10.484668 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5840b-a815-4adf-b818-733aa26ce645 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:44:10.494761 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 764/1527] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:44:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:44:14.453314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac866d-6d6f-43aa-b666-239984e6e7a0 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:14.454825 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac866d-6d6f-43aa-b666-239984e6e7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:14.455120 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac866d-6d6f-43aa-b666-239984e6e7a0 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:14.456567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 08:44:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:44:14.462388 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f173bb-a4df-4697-a289-a2487b4c9bae None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:14.462847 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f173bb-a4df-4697-a289-a2487b4c9bae None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:14.462949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f173bb-a4df-4697-a289-a2487b4c9bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:14.601632 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-17f173bb-a4df-4697-a289-a2487b4c9bae None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:44:14.602766 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-17f173bb-a4df-4697-a289-a2487b4c9bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:44:14.611903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 08:44:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:44:14.650140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac866d-6d6f-43aa-b666-239984e6e7a0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:44:14.651928 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-19ac866d-6d6f-43aa-b666-239984e6e7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:44:14.682502 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:44:14.713108 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac866d-6d6f-43aa-b666-239984e6e7a0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:44:14.716677 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb5736-7cfb-4659-85b4-0196a8b4af5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:44:14.717322 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb5736-7cfb-4659-85b4-0196a8b4af5d service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:14.717606 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb5736-7cfb-4659-85b4-0196a8b4af5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:14.717880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb5736-7cfb-4659-85b4-0196a8b4af5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:14.728258 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb5736-7cfb-4659-85b4-0196a8b4af5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:44:14.729667 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb5736-7cfb-4659-85b4-0196a8b4af5d service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:44:14.739889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 765/1530] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:44:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:44:16.754473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2d6a5-97a2-4dfd-b0f2-bf6457954026 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:16.755040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2d6a5-97a2-4dfd-b0f2-bf6457954026 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:16.755040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2d6a5-97a2-4dfd-b0f2-bf6457954026 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:16.756433 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 08:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:44:16.761703 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96d1cb-a51c-4004-a49f-d94e0d25f8a6 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:16.762112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96d1cb-a51c-4004-a49f-d94e0d25f8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:16.762259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96d1cb-a51c-4004-a49f-d94e0d25f8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:16.780704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-ba96d1cb-a51c-4004-a49f-d94e0d25f8a6 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:44:16.782713 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ba96d1cb-a51c-4004-a49f-d94e0d25f8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:44:16.790727 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 08:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:44:16.826605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2d6a5-97a2-4dfd-b0f2-bf6457954026 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:44:16.839318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2d6a5-97a2-4dfd-b0f2-bf6457954026 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:44:16.842520 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd205e3-5905-40fc-a53e-813920322fd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:44:16.843168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd205e3-5905-40fc-a53e-813920322fd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:16.843450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd205e3-5905-40fc-a53e-813920322fd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:16.843761 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd205e3-5905-40fc-a53e-813920322fd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:16.854107 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd205e3-5905-40fc-a53e-813920322fd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:44:16.855558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd205e3-5905-40fc-a53e-813920322fd9 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:44:16.865180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 767/1533] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:44:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:44:20.389783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2838694-63b2-4afd-881d-3001526a1ae1 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:20.390413 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2838694-63b2-4afd-881d-3001526a1ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:20.390413 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2838694-63b2-4afd-881d-3001526a1ae1 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:20.392363 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 08:44:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:44:20.397742 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49477c-a1f6-4b8b-aaf8-515b99985161 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:20.398201 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49477c-a1f6-4b8b-aaf8-515b99985161 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:20.398317 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49477c-a1f6-4b8b-aaf8-515b99985161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:20.416802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3b49477c-a1f6-4b8b-aaf8-515b99985161 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:44:20.418012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3b49477c-a1f6-4b8b-aaf8-515b99985161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:44:20.426744 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 08:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:44:20.734226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2838694-63b2-4afd-881d-3001526a1ae1 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:44:20.736609 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d2838694-63b2-4afd-881d-3001526a1ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:44:20.794288 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2838694-63b2-4afd-881d-3001526a1ae1 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:44:20.798540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50377e04-9d9d-40ca-992f-86c4451ca4fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:44:20.799193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50377e04-9d9d-40ca-992f-86c4451ca4fb service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:20.799479 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50377e04-9d9d-40ca-992f-86c4451ca4fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:20.799788 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50377e04-9d9d-40ca-992f-86c4451ca4fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:20.809072 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50377e04-9d9d-40ca-992f-86c4451ca4fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:44:20.810498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50377e04-9d9d-40ca-992f-86c4451ca4fb service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:44:20.820174 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 768/1536] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:44:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:44:22.853807 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d8054-acfb-4897-afb4-f8c87bc0a492 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:22.854339 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d8054-acfb-4897-afb4-f8c87bc0a492 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:22.854447 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d8054-acfb-4897-afb4-f8c87bc0a492 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:22.856022 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 08:44:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:44:22.862039 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed190c-a61b-402e-8e24-9bcbe05b89a2 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:22.862484 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed190c-a61b-402e-8e24-9bcbe05b89a2 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:22.862646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed190c-a61b-402e-8e24-9bcbe05b89a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:22.882476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-44ed190c-a61b-402e-8e24-9bcbe05b89a2 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:44:22.883741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-44ed190c-a61b-402e-8e24-9bcbe05b89a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:44:22.891979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 08:44:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:44:23.201358 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d8054-acfb-4897-afb4-f8c87bc0a492 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:44:23.203205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-955d8054-acfb-4897-afb4-f8c87bc0a492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:44:23.217064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d8054-acfb-4897-afb4-f8c87bc0a492 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:44:23.221325 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c271cd-0fcf-4287-bca3-77d18b3585af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:44:23.221978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c271cd-0fcf-4287-bca3-77d18b3585af service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:44:23.222289 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c271cd-0fcf-4287-bca3-77d18b3585af service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:44:23.222567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c271cd-0fcf-4287-bca3-77d18b3585af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:44:23.232496 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c271cd-0fcf-4287-bca3-77d18b3585af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:44:23.233977 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c271cd-0fcf-4287-bca3-77d18b3585af service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:44:23.244658 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 770/1539] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:44:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:44:32.876588 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=47 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:44:33.240215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=82 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:45:52.583720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b9403-96ca-4591-9c72-6bb1ceb09498 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:52.586397 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b9403-96ca-4591-9c72-6bb1ceb09498 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:52.586397 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b9403-96ca-4591-9c72-6bb1ceb09498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:52.591159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a577c3-c41b-48e8-83c8-2034ecaa6909 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:52.592959 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a577c3-c41b-48e8-83c8-2034ecaa6909 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:52.593664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a577c3-c41b-48e8-83c8-2034ecaa6909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:52.628883 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-bc4b9403-96ca-4591-9c72-6bb1ceb09498 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:52.629154 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f0a577c3-c41b-48e8-83c8-2034ecaa6909 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:52.630156 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bc4b9403-96ca-4591-9c72-6bb1ceb09498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:52.630381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f0a577c3-c41b-48e8-83c8-2034ecaa6909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:52.647531 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 770/1540] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:52.649544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 771/1541] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:45:52.650218 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcee41-3d82-4754-b56a-98cdd239fb36 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:52.650577 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcee41-3d82-4754-b56a-98cdd239fb36 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:52.650861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcee41-3d82-4754-b56a-98cdd239fb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:52.658973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47647e-5464-4ad4-91c6-1d9119cff26c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:52.659322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47647e-5464-4ad4-91c6-1d9119cff26c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:52.659538 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47647e-5464-4ad4-91c6-1d9119cff26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:52.689099 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-04bcee41-3d82-4754-b56a-98cdd239fb36 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:52.690401 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-04bcee41-3d82-4754-b56a-98cdd239fb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:52.693465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-cd47647e-5464-4ad4-91c6-1d9119cff26c None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:52.694938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cd47647e-5464-4ad4-91c6-1d9119cff26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:52.707492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 771/1542] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:45:52.709167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55477c78-3b1b-4df1-ba97-74791376b34d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:52.709454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55477c78-3b1b-4df1-ba97-74791376b34d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:52.709727 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55477c78-3b1b-4df1-ba97-74791376b34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:52.713435 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 772/1543] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:52.716181 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de971965-8f0f-431d-a9e3-39d5203f862f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:52.716480 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de971965-8f0f-431d-a9e3-39d5203f862f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:52.716757 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de971965-8f0f-431d-a9e3-39d5203f862f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:52.745033 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-55477c78-3b1b-4df1-ba97-74791376b34d None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:52.745273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-de971965-8f0f-431d-a9e3-39d5203f862f None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:52.746274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-55477c78-3b1b-4df1-ba97-74791376b34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:52.746758 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-de971965-8f0f-431d-a9e3-39d5203f862f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:52.770112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 773/1544] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:45:52.771003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de971965-8f0f-431d-a9e3-39d5203f862f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:52.782801 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 772/1545] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:52.783725 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55477c78-3b1b-4df1-ba97-74791376b34d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:52.789698 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de971965-8f0f-431d-a9e3-39d5203f862f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:52.798889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55477c78-3b1b-4df1-ba97-74791376b34d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:52.802995 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e20225-c5dd-4baf-8280-d3717fc7f787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:52.803700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e20225-c5dd-4baf-8280-d3717fc7f787 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:52.804011 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e20225-c5dd-4baf-8280-d3717fc7f787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:52.804284 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e20225-c5dd-4baf-8280-d3717fc7f787 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:52.805026 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e20225-c5dd-4baf-8280-d3717fc7f787 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:45:52.805330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e20225-c5dd-4baf-8280-d3717fc7f787 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:52.806363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-24e20225-c5dd-4baf-8280-d3717fc7f787 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 08:45:52.806363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Jul 18 08:45:52.806363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: Jul 18 08:45:52.807260 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e20225-c5dd-4baf-8280-d3717fc7f787 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:52.817353 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592f6ae-95f7-41df-8f9d-db3ec2369375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:52.818416 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592f6ae-95f7-41df-8f9d-db3ec2369375 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:52.818965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592f6ae-95f7-41df-8f9d-db3ec2369375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:52.819515 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592f6ae-95f7-41df-8f9d-db3ec2369375 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:52.820778 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592f6ae-95f7-41df-8f9d-db3ec2369375 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:45:52.821332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592f6ae-95f7-41df-8f9d-db3ec2369375 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:52.821622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 774/1546] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:45:52.822489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e20225-c5dd-4baf-8280-d3717fc7f787 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:52.823926 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592f6ae-95f7-41df-8f9d-db3ec2369375 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:52.849987 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e20225-c5dd-4baf-8280-d3717fc7f787 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:52.855864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 773/1547] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:45:52.857096 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592f6ae-95f7-41df-8f9d-db3ec2369375 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:52.864089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f440e-3d2c-4a0b-bf9c-9353c652febe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:52.864964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f440e-3d2c-4a0b-bf9c-9353c652febe admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:52.865253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f440e-3d2c-4a0b-bf9c-9353c652febe admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:52.865711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f440e-3d2c-4a0b-bf9c-9353c652febe admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:52.866458 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f440e-3d2c-4a0b-bf9c-9353c652febe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:45:52.866771 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f440e-3d2c-4a0b-bf9c-9353c652febe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:52.868438 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f440e-3d2c-4a0b-bf9c-9353c652febe admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:52.888398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592f6ae-95f7-41df-8f9d-db3ec2369375 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:52.896439 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 775/1548] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:45:52.897570 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f440e-3d2c-4a0b-bf9c-9353c652febe admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:52.906892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b48d1-3f91-4958-b2f4-866a9f6085cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:52.908005 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b48d1-3f91-4958-b2f4-866a9f6085cb admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:52.908549 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b48d1-3f91-4958-b2f4-866a9f6085cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:52.909049 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b48d1-3f91-4958-b2f4-866a9f6085cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:52.909972 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b48d1-3f91-4958-b2f4-866a9f6085cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:52.911226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: WARNING py.warnings [None req-8a6b48d1-3f91-4958-b2f4-866a9f6085cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 08:45:52.911226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: warnings.warn(msg) Jul 18 08:45:52.911226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: Jul 18 08:45:52.912856 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b48d1-3f91-4958-b2f4-866a9f6085cb admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:52.919694 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f440e-3d2c-4a0b-bf9c-9353c652febe admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:52.937585 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a560bb-ff35-4508-bce8-bc46a0b9371a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:52.938269 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a560bb-ff35-4508-bce8-bc46a0b9371a admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:52.938554 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a560bb-ff35-4508-bce8-bc46a0b9371a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:52.938832 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a560bb-ff35-4508-bce8-bc46a0b9371a admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:52.939528 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a560bb-ff35-4508-bce8-bc46a0b9371a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:52.946885 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a560bb-ff35-4508-bce8-bc46a0b9371a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:53.075730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 774/1549] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:45:52 2022] POST /identity/v3/projects => generated 385 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:53.076778 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b48d1-3f91-4958-b2f4-866a9f6085cb admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:53.079376 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 776/1550] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:45:52 2022] POST /identity/v3/projects => generated 365 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:53.081000 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de8bc7-a5a0-487d-9d74-8b54c681861b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.081350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de8bc7-a5a0-487d-9d74-8b54c681861b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.081624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de8bc7-a5a0-487d-9d74-8b54c681861b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.108439 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-15de8bc7-a5a0-487d-9d74-8b54c681861b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:53.110048 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-15de8bc7-a5a0-487d-9d74-8b54c681861b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:53.156323 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b48d1-3f91-4958-b2f4-866a9f6085cb admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:53.172861 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 777/1551] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:53.173910 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df3848-26f5-4718-aca1-aae812666189 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.174344 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df3848-26f5-4718-aca1-aae812666189 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.174636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df3848-26f5-4718-aca1-aae812666189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.197871 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f5df3848-26f5-4718-aca1-aae812666189 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:53.199099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f5df3848-26f5-4718-aca1-aae812666189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:53.207597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 778/1552] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:53.208911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbeceb3-9cd9-49b7-bf9d-436cb66b5443 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.209208 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbeceb3-9cd9-49b7-bf9d-436cb66b5443 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.209490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbeceb3-9cd9-49b7-bf9d-436cb66b5443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.222781 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c6bec-ebfe-4009-ae41-ada51653148e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:53.223445 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c6bec-ebfe-4009-ae41-ada51653148e admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.223777 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c6bec-ebfe-4009-ae41-ada51653148e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.224063 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c6bec-ebfe-4009-ae41-ada51653148e admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.224759 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c6bec-ebfe-4009-ae41-ada51653148e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:53.226704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c6bec-ebfe-4009-ae41-ada51653148e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:53.230770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-9bbeceb3-9cd9-49b7-bf9d-436cb66b5443 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:53.232057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9bbeceb3-9cd9-49b7-bf9d-436cb66b5443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:53.242018 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 779/1553] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:53.242620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbeceb3-9cd9-49b7-bf9d-436cb66b5443 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:53.254054 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbeceb3-9cd9-49b7-bf9d-436cb66b5443 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:53.258710 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfc37f-f782-40ec-b29a-d7c00ab18a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:53.259371 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfc37f-f782-40ec-b29a-d7c00ab18a90 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.259651 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfc37f-f782-40ec-b29a-d7c00ab18a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.259963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfc37f-f782-40ec-b29a-d7c00ab18a90 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.260662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbfc37f-f782-40ec-b29a-d7c00ab18a90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:53.262532 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-bcbfc37f-f782-40ec-b29a-d7c00ab18a90 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 08:45:53.262532 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Jul 18 08:45:53.262532 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: Jul 18 08:45:53.263714 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbfc37f-f782-40ec-b29a-d7c00ab18a90 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:53.324301 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 775/1554] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:45:53 2022] POST /identity/v3/projects => generated 379 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:53.324942 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c6bec-ebfe-4009-ae41-ada51653148e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:53.367826 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:45:53.387796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c6bec-ebfe-4009-ae41-ada51653148e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:53.393070 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e364a00-f275-4b4d-b2ff-77dba1edb7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:53.393808 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e364a00-f275-4b4d-b2ff-77dba1edb7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.394027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e364a00-f275-4b4d-b2ff-77dba1edb7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.394303 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e364a00-f275-4b4d-b2ff-77dba1edb7c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.395047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e364a00-f275-4b4d-b2ff-77dba1edb7c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:53.397012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e364a00-f275-4b4d-b2ff-77dba1edb7c2 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:53.404167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 780/1555] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:45:53 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:53.413186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebde2fd-9a84-4158-a1b1-966af8f850ea None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.413186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebde2fd-9a84-4158-a1b1-966af8f850ea None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.413186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebde2fd-9a84-4158-a1b1-966af8f850ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.430836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-eebde2fd-9a84-4158-a1b1-966af8f850ea None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:53.432282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-eebde2fd-9a84-4158-a1b1-966af8f850ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:53.441117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 781/1556] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:53.442793 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf1a82-9d57-4e24-8deb-b7c34b4d8e53 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.443089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf1a82-9d57-4e24-8deb-b7c34b4d8e53 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.443367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf1a82-9d57-4e24-8deb-b7c34b4d8e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.463133 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-6daf1a82-9d57-4e24-8deb-b7c34b4d8e53 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:53.464410 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6daf1a82-9d57-4e24-8deb-b7c34b4d8e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:53.473797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 782/1557] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:53.475327 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bf902-8009-43fe-a4e1-6548bbb6f7bf None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.475723 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bf902-8009-43fe-a4e1-6548bbb6f7bf None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.476026 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bf902-8009-43fe-a4e1-6548bbb6f7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.496607 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-542bf902-8009-43fe-a4e1-6548bbb6f7bf None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:53.498013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-542bf902-8009-43fe-a4e1-6548bbb6f7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:53.506450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 783/1558] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:53.507908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e80166-07ba-4706-8e38-21d1a2fa5237 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.508205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e80166-07ba-4706-8e38-21d1a2fa5237 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.508484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e80166-07ba-4706-8e38-21d1a2fa5237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.533677 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-22e80166-07ba-4706-8e38-21d1a2fa5237 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:53.533677 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-22e80166-07ba-4706-8e38-21d1a2fa5237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:53.535991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 776/1559] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:45:53 2022] POST /identity/v3/users => generated 283 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:53.541390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 784/1560] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:53.541625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb54573-9176-4093-9dc0-8d3fcff07ee8 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.541625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb54573-9176-4093-9dc0-8d3fcff07ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.541625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb54573-9176-4093-9dc0-8d3fcff07ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.542006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e80166-07ba-4706-8e38-21d1a2fa5237 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:53.551023 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e80166-07ba-4706-8e38-21d1a2fa5237 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:53.556352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff4416-f67c-4e38-b427-b17b667bbd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:53.557030 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff4416-f67c-4e38-b427-b17b667bbd9c admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.557311 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff4416-f67c-4e38-b427-b17b667bbd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.557584 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff4416-f67c-4e38-b427-b17b667bbd9c admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.557783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-bbb54573-9176-4093-9dc0-8d3fcff07ee8 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:53.558279 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff4416-f67c-4e38-b427-b17b667bbd9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:53.558967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bbb54573-9176-4093-9dc0-8d3fcff07ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:53.560245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff4416-f67c-4e38-b427-b17b667bbd9c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:53.567256 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 777/1561] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:53.568533 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb54573-9176-4093-9dc0-8d3fcff07ee8 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:53.579328 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb54573-9176-4093-9dc0-8d3fcff07ee8 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:53.583458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1f7e3-c0a6-43ed-897e-8b8abe86e3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:53.584168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1f7e3-c0a6-43ed-897e-8b8abe86e3ea admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.584819 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1f7e3-c0a6-43ed-897e-8b8abe86e3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.584819 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1f7e3-c0a6-43ed-897e-8b8abe86e3ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.585433 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1f7e3-c0a6-43ed-897e-8b8abe86e3ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:53.587061 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1f7e3-c0a6-43ed-897e-8b8abe86e3ea admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:53.598084 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 778/1562] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:53.599478 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc766669-bafc-4b78-b042-6b26645c0b37 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.599801 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc766669-bafc-4b78-b042-6b26645c0b37 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.600086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc766669-bafc-4b78-b042-6b26645c0b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.618724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-bc766669-bafc-4b78-b042-6b26645c0b37 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:53.619909 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bc766669-bafc-4b78-b042-6b26645c0b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:53.628227 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 779/1563] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:53.629619 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a08f82-35cf-4368-9296-1883bb9c83cf None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.629915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a08f82-35cf-4368-9296-1883bb9c83cf None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.630196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a08f82-35cf-4368-9296-1883bb9c83cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.650117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-13a08f82-35cf-4368-9296-1883bb9c83cf None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:53.651312 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-13a08f82-35cf-4368-9296-1883bb9c83cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:53.660175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 780/1564] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:53.663957 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a08f82-35cf-4368-9296-1883bb9c83cf None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:53.668203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 785/1565] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:45:53 2022] POST /identity/v3/users => generated 290 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:53.668203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff4416-f67c-4e38-b427-b17b667bbd9c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:53.672907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a08f82-35cf-4368-9296-1883bb9c83cf None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:53.676027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52e5c2-e1ba-4554-8fb2-96f0ce383fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:53.676695 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52e5c2-e1ba-4554-8fb2-96f0ce383fce admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.676971 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52e5c2-e1ba-4554-8fb2-96f0ce383fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.677242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52e5c2-e1ba-4554-8fb2-96f0ce383fce admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.677967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52e5c2-e1ba-4554-8fb2-96f0ce383fce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:45:53.678273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52e5c2-e1ba-4554-8fb2-96f0ce383fce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:53.679460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff4416-f67c-4e38-b427-b17b667bbd9c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:53.679954 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52e5c2-e1ba-4554-8fb2-96f0ce383fce admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:53.684682 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2855bdc-ece5-49c5-8a41-5fe51c512ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:53.685367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2855bdc-ece5-49c5-8a41-5fe51c512ddd admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.685674 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2855bdc-ece5-49c5-8a41-5fe51c512ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.685960 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2855bdc-ece5-49c5-8a41-5fe51c512ddd admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.686742 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2855bdc-ece5-49c5-8a41-5fe51c512ddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:53.688431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2855bdc-ece5-49c5-8a41-5fe51c512ddd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:53.692488 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 781/1566] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:45:53.693401 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52e5c2-e1ba-4554-8fb2-96f0ce383fce admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:53.698916 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 786/1567] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:53.699802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2855bdc-ece5-49c5-8a41-5fe51c512ddd admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:53.711259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52e5c2-e1ba-4554-8fb2-96f0ce383fce admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:53.711527 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2855bdc-ece5-49c5-8a41-5fe51c512ddd admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:53.716180 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084041cd-bedb-418e-b92c-afa88f3fb00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:53.716441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536114d9-613d-42d5-9d5b-586223fd9d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:53.716862 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084041cd-bedb-418e-b92c-afa88f3fb00c admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.716997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536114d9-613d-42d5-9d5b-586223fd9d9a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.717139 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084041cd-bedb-418e-b92c-afa88f3fb00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.717269 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536114d9-613d-42d5-9d5b-586223fd9d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.717985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084041cd-bedb-418e-b92c-afa88f3fb00c admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.718078 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536114d9-613d-42d5-9d5b-586223fd9d9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.718819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536114d9-613d-42d5-9d5b-586223fd9d9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:45:53.719114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084041cd-bedb-418e-b92c-afa88f3fb00c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:45:53.719253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536114d9-613d-42d5-9d5b-586223fd9d9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:53.719434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084041cd-bedb-418e-b92c-afa88f3fb00c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:53.720927 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536114d9-613d-42d5-9d5b-586223fd9d9a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:53.721048 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084041cd-bedb-418e-b92c-afa88f3fb00c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:53.722885 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:45:53.734427 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 782/1568] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:45:53.735282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084041cd-bedb-418e-b92c-afa88f3fb00c admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:53.737165 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 787/1569] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:45:53.738000 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536114d9-613d-42d5-9d5b-586223fd9d9a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:53.748098 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536114d9-613d-42d5-9d5b-586223fd9d9a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:53.752878 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72675a7-75f1-451f-aaa3-38104d337574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:53.753543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72675a7-75f1-451f-aaa3-38104d337574 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.753825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72675a7-75f1-451f-aaa3-38104d337574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.754099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72675a7-75f1-451f-aaa3-38104d337574 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.754836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72675a7-75f1-451f-aaa3-38104d337574 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:45:53.755148 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72675a7-75f1-451f-aaa3-38104d337574 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:53.757211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72675a7-75f1-451f-aaa3-38104d337574 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:53.757335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084041cd-bedb-418e-b92c-afa88f3fb00c admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:53.762070 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e6e92-b88d-4bf6-9089-06a56b57ca88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:53.762736 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e6e92-b88d-4bf6-9089-06a56b57ca88 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.763021 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e6e92-b88d-4bf6-9089-06a56b57ca88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.763297 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e6e92-b88d-4bf6-9089-06a56b57ca88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.764582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e6e92-b88d-4bf6-9089-06a56b57ca88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:53.766277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e6e92-b88d-4bf6-9089-06a56b57ca88 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:53.770137 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 788/1570] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:45:53.773427 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72675a7-75f1-451f-aaa3-38104d337574 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:53.777308 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 783/1571] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:53.778238 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e6e92-b88d-4bf6-9089-06a56b57ca88 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:53.784527 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72675a7-75f1-451f-aaa3-38104d337574 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:53.789320 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9759d3-3adf-40bc-a64a-d1771adaf4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:53.790134 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9759d3-3adf-40bc-a64a-d1771adaf4fc admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.790364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e6e92-b88d-4bf6-9089-06a56b57ca88 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:53.790733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9759d3-3adf-40bc-a64a-d1771adaf4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.791186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9759d3-3adf-40bc-a64a-d1771adaf4fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.792339 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9759d3-3adf-40bc-a64a-d1771adaf4fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:45:53.792655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9759d3-3adf-40bc-a64a-d1771adaf4fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:53.794277 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9759d3-3adf-40bc-a64a-d1771adaf4fc admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:53.795010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc78fda-d3bb-4de4-953c-e6546bee1016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:53.795832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc78fda-d3bb-4de4-953c-e6546bee1016 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.796128 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc78fda-d3bb-4de4-953c-e6546bee1016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.796410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc78fda-d3bb-4de4-953c-e6546bee1016 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.797157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc78fda-d3bb-4de4-953c-e6546bee1016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:53.798793 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc78fda-d3bb-4de4-953c-e6546bee1016 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:53.807876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 789/1572] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:45:53.808705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9759d3-3adf-40bc-a64a-d1771adaf4fc admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:53.813245 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 784/1573] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:53.813245 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc78fda-d3bb-4de4-953c-e6546bee1016 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:53.820169 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9759d3-3adf-40bc-a64a-d1771adaf4fc admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:53.823217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc78fda-d3bb-4de4-953c-e6546bee1016 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:53.824932 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593d36c-00df-445d-a02e-b42c3badb29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:53.825673 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593d36c-00df-445d-a02e-b42c3badb29c admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.825967 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593d36c-00df-445d-a02e-b42c3badb29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.826243 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593d36c-00df-445d-a02e-b42c3badb29c admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.827549 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593d36c-00df-445d-a02e-b42c3badb29c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:53.828883 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb62ea-7684-461f-a28a-d79d7c95851b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:53.829547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb62ea-7684-461f-a28a-d79d7c95851b admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.829832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb62ea-7684-461f-a28a-d79d7c95851b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.830060 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593d36c-00df-445d-a02e-b42c3badb29c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:53.830166 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb62ea-7684-461f-a28a-d79d7c95851b admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.830820 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb62ea-7684-461f-a28a-d79d7c95851b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:53.832452 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb62ea-7684-461f-a28a-d79d7c95851b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:53.845382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 785/1574] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:53.846225 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb62ea-7684-461f-a28a-d79d7c95851b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:53.856326 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb62ea-7684-461f-a28a-d79d7c95851b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:53.861431 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c7d7d-f822-4332-8dc6-5eafcfdd9bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:53.862097 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c7d7d-f822-4332-8dc6-5eafcfdd9bfd admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:53.862383 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c7d7d-f822-4332-8dc6-5eafcfdd9bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:53.862659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c7d7d-f822-4332-8dc6-5eafcfdd9bfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:53.863358 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c7d7d-f822-4332-8dc6-5eafcfdd9bfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:53.865363 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c7d7d-f822-4332-8dc6-5eafcfdd9bfd admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:53.908144 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 790/1575] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:45:53 2022] POST /identity/v3/projects => generated 403 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:53.908894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593d36c-00df-445d-a02e-b42c3badb29c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:53.953374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 786/1576] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:45:53 2022] POST /identity/v3/projects => generated 361 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:53.954438 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c7d7d-f822-4332-8dc6-5eafcfdd9bfd admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:53.978459 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593d36c-00df-445d-a02e-b42c3badb29c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:54.019802 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c7d7d-f822-4332-8dc6-5eafcfdd9bfd admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:54.024616 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f679a-0020-48cd-bd42-a4eff756c184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:54.025288 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f679a-0020-48cd-bd42-a4eff756c184 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:54.025580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f679a-0020-48cd-bd42-a4eff756c184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:54.025859 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f679a-0020-48cd-bd42-a4eff756c184 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:54.026558 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f679a-0020-48cd-bd42-a4eff756c184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:54.028554 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f679a-0020-48cd-bd42-a4eff756c184 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:54.039839 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da7604-a599-4524-b2ba-ee5f35f05dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:54.040515 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da7604-a599-4524-b2ba-ee5f35f05dab admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:54.040804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da7604-a599-4524-b2ba-ee5f35f05dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:54.041084 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da7604-a599-4524-b2ba-ee5f35f05dab admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:54.041827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da7604-a599-4524-b2ba-ee5f35f05dab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:54.043904 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da7604-a599-4524-b2ba-ee5f35f05dab admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:54.135493 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:45:54.142849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 787/1577] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:45:53 2022] POST /identity/v3/projects => generated 385 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:54.143657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f679a-0020-48cd-bd42-a4eff756c184 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:54.165674 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 791/1578] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:45:53 2022] POST /identity/v3/projects => generated 391 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:54.165674 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da7604-a599-4524-b2ba-ee5f35f05dab admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:54.210903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f679a-0020-48cd-bd42-a4eff756c184 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:54.216973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:45:54.239347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da7604-a599-4524-b2ba-ee5f35f05dab admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:54.244221 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3b7ba-4f01-49ec-bb00-c6120d8d4696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:54.244897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3b7ba-4f01-49ec-bb00-c6120d8d4696 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:54.245187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3b7ba-4f01-49ec-bb00-c6120d8d4696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:54.245612 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3b7ba-4f01-49ec-bb00-c6120d8d4696 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:54.246358 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3b7ba-4f01-49ec-bb00-c6120d8d4696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:54.248075 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3b7ba-4f01-49ec-bb00-c6120d8d4696 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:54.262257 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 792/1579] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:54.263419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3b7ba-4f01-49ec-bb00-c6120d8d4696 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:54.272989 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa7d1f-4ee8-46a0-a265-f8cb5358dc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:54.273895 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa7d1f-4ee8-46a0-a265-f8cb5358dc6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:54.274178 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa7d1f-4ee8-46a0-a265-f8cb5358dc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:54.274454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa7d1f-4ee8-46a0-a265-f8cb5358dc6c admin admin] PATH_INFO: `/v3/projects/ef2947427ce24f46baad0172db285cc8/users/88b4bd52ede644f0aac8dff94e09dd7d/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:54.276245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3b7ba-4f01-49ec-bb00-c6120d8d4696 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:54.281362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525653c-cf05-4c35-a28a-24a6dab41999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:54.282279 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525653c-cf05-4c35-a28a-24a6dab41999 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:54.282540 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525653c-cf05-4c35-a28a-24a6dab41999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:54.283383 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525653c-cf05-4c35-a28a-24a6dab41999 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:54.284215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2525653c-cf05-4c35-a28a-24a6dab41999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:54.286079 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2525653c-cf05-4c35-a28a-24a6dab41999 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:54.290373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa7d1f-4ee8-46a0-a265-f8cb5358dc6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2947427ce24f46baad0172db285cc8, user_id=88b4bd52ede644f0aac8dff94e09dd7d, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:54.296323 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: WARNING py.warnings [None req-f0fa7d1f-4ee8-46a0-a265-f8cb5358dc6c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 08:45:54.296323 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: warnings.warn(msg) Jul 18 08:45:54.296323 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: Jul 18 08:45:54.296323 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa7d1f-4ee8-46a0-a265-f8cb5358dc6c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:54.380581 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 788/1580] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:45:54 2022] PUT /identity/v3/projects/ef2947427ce24f46baad0172db285cc8/users/88b4bd52ede644f0aac8dff94e09dd7d/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:45:54.381406 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa7d1f-4ee8-46a0-a265-f8cb5358dc6c admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:54.393891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 793/1581] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:45:54 2022] POST /identity/v3/projects => generated 375 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:54.394761 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525653c-cf05-4c35-a28a-24a6dab41999 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:54.454118 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa7d1f-4ee8-46a0-a265-f8cb5358dc6c admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:54.459037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcfa48-7e19-430e-9935-590ceaede723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:54.459982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcfa48-7e19-430e-9935-590ceaede723 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:54.460270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcfa48-7e19-430e-9935-590ceaede723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:54.460551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcfa48-7e19-430e-9935-590ceaede723 admin admin] PATH_INFO: `/v3/projects/a5746e9b1fe24e929b4d3355b6605423/users/91c81496c57c471ea3858579b3aa83dc/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:54.465814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525653c-cf05-4c35-a28a-24a6dab41999 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:54.472184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a1e75-2fc3-4e6c-b06f-dec18db85c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:54.473048 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a1e75-2fc3-4e6c-b06f-dec18db85c93 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:54.473170 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a1e75-2fc3-4e6c-b06f-dec18db85c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:54.473450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a1e75-2fc3-4e6c-b06f-dec18db85c93 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:54.474149 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a1e75-2fc3-4e6c-b06f-dec18db85c93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:54.476226 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a1e75-2fc3-4e6c-b06f-dec18db85c93 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:54.479295 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dcfa48-7e19-430e-9935-590ceaede723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5746e9b1fe24e929b4d3355b6605423, user_id=91c81496c57c471ea3858579b3aa83dc, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:54.482120 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dcfa48-7e19-430e-9935-590ceaede723 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:54.574281 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 789/1582] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:45:54 2022] PUT /identity/v3/projects/a5746e9b1fe24e929b4d3355b6605423/users/91c81496c57c471ea3858579b3aa83dc/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:45:54.574706 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 794/1583] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:45:54 2022] POST /identity/v3/users => generated 302 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:54.575544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a1e75-2fc3-4e6c-b06f-dec18db85c93 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:54.575919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcfa48-7e19-430e-9935-590ceaede723 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:54.589554 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a1e75-2fc3-4e6c-b06f-dec18db85c93 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:54.596670 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:45:54.648243 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcfa48-7e19-430e-9935-590ceaede723 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:54.650113 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ec479-6a31-4419-b45f-35bb3bf6b828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:54.650803 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ec479-6a31-4419-b45f-35bb3bf6b828 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:54.651087 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ec479-6a31-4419-b45f-35bb3bf6b828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:54.651399 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ec479-6a31-4419-b45f-35bb3bf6b828 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:54.652438 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ec479-6a31-4419-b45f-35bb3bf6b828 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:54.653321 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1db5e6-2353-4f00-ba64-f9d2257401b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:54.654227 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1db5e6-2353-4f00-ba64-f9d2257401b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:54.654411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ec479-6a31-4419-b45f-35bb3bf6b828 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:54.655690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1db5e6-2353-4f00-ba64-f9d2257401b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:54.656100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1db5e6-2353-4f00-ba64-f9d2257401b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:54.656909 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1db5e6-2353-4f00-ba64-f9d2257401b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:54.658833 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1db5e6-2353-4f00-ba64-f9d2257401b2 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:54.769484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 795/1584] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:45:54 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:54.770211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ec479-6a31-4419-b45f-35bb3bf6b828 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:54.780825 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 790/1585] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:45:54 2022] POST /identity/v3/users => generated 281 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:54.781049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ec479-6a31-4419-b45f-35bb3bf6b828 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:54.781646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1db5e6-2353-4f00-ba64-f9d2257401b2 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:54.788728 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8639e9-bcba-412a-bf17-423175d8085c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:54.789401 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8639e9-bcba-412a-bf17-423175d8085c admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:54.789725 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8639e9-bcba-412a-bf17-423175d8085c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:54.790002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8639e9-bcba-412a-bf17-423175d8085c admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:54.790694 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8639e9-bcba-412a-bf17-423175d8085c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:54.791788 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1db5e6-2353-4f00-ba64-f9d2257401b2 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:54.792694 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8639e9-bcba-412a-bf17-423175d8085c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:54.797283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ea94a-fef5-4bda-9782-194e6b118050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:54.802019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ea94a-fef5-4bda-9782-194e6b118050 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:54.802019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ea94a-fef5-4bda-9782-194e6b118050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:54.802019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ea94a-fef5-4bda-9782-194e6b118050 admin admin] PATH_INFO: `/v3/projects/4a773b75a4b4442ca01dc2b0d9e4e5d9/users/67a5cc6144c0434b9f16fb050c4c7662/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:54.815498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ea94a-fef5-4bda-9782-194e6b118050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a773b75a4b4442ca01dc2b0d9e4e5d9, user_id=67a5cc6144c0434b9f16fb050c4c7662, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:54.818522 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ea94a-fef5-4bda-9782-194e6b118050 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:54.883158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 796/1586] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:45:54 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:54.884499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc04a92-ef68-48e3-bb6c-fea4c692d3f8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:54.884792 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc04a92-ef68-48e3-bb6c-fea4c692d3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:54.885078 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc04a92-ef68-48e3-bb6c-fea4c692d3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:54.903304 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ccc04a92-ef68-48e3-bb6c-fea4c692d3f8 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:54.904570 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ccc04a92-ef68-48e3-bb6c-fea4c692d3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:54.911958 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 791/1587] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:45:54 2022] PUT /identity/v3/projects/4a773b75a4b4442ca01dc2b0d9e4e5d9/users/67a5cc6144c0434b9f16fb050c4c7662/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:45:54.915269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ea94a-fef5-4bda-9782-194e6b118050 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:54.915400 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:45:54.970246 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 797/1588] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:54.971106 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc04a92-ef68-48e3-bb6c-fea4c692d3f8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:54.978914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ea94a-fef5-4bda-9782-194e6b118050 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:54.980660 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc04a92-ef68-48e3-bb6c-fea4c692d3f8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:54.983964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37eb42-a9b5-4982-b33c-9a9883bf5ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:54.988202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37eb42-a9b5-4982-b33c-9a9883bf5ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:54.988202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37eb42-a9b5-4982-b33c-9a9883bf5ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:54.988202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37eb42-a9b5-4982-b33c-9a9883bf5ba6 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:54.988503 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21cd81-f7c8-43c4-bf64-dfd9ed4ac5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:54.988611 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37eb42-a9b5-4982-b33c-9a9883bf5ba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:54.988711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21cd81-f7c8-43c4-bf64-dfd9ed4ac5a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:54.988711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21cd81-f7c8-43c4-bf64-dfd9ed4ac5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:54.988711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21cd81-f7c8-43c4-bf64-dfd9ed4ac5a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:54.988711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21cd81-f7c8-43c4-bf64-dfd9ed4ac5a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:54.989061 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37eb42-a9b5-4982-b33c-9a9883bf5ba6 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:54.989574 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21cd81-f7c8-43c4-bf64-dfd9ed4ac5a5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:55.000746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 798/1589] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:55.002068 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c5fdf-20af-47ff-9868-516726ac0453 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:55.002570 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c5fdf-20af-47ff-9868-516726ac0453 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:55.003029 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c5fdf-20af-47ff-9868-516726ac0453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:55.031821 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3a7c5fdf-20af-47ff-9868-516726ac0453 None None] MFA Rules not processed for user `91c81496c57c471ea3858579b3aa83dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:55.032952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3a7c5fdf-20af-47ff-9868-516726ac0453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:55.077918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 792/1590] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:45:54 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:55.078775 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37eb42-a9b5-4982-b33c-9a9883bf5ba6 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:55.090847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37eb42-a9b5-4982-b33c-9a9883bf5ba6 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:55.097045 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e7cbf-2825-4ff0-af34-dd777b6d9504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:55.097721 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e7cbf-2825-4ff0-af34-dd777b6d9504 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:55.098007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e7cbf-2825-4ff0-af34-dd777b6d9504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:55.099376 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e7cbf-2825-4ff0-af34-dd777b6d9504 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:55.099482 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 799/1591] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:55 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:55.099567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e7cbf-2825-4ff0-af34-dd777b6d9504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:55.099642 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c5fdf-20af-47ff-9868-516726ac0453 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:55.100815 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e7cbf-2825-4ff0-af34-dd777b6d9504 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:55.111330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c5fdf-20af-47ff-9868-516726ac0453 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:55.113047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 793/1592] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:55.113974 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e7cbf-2825-4ff0-af34-dd777b6d9504 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:55.117638 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f340d-d344-4451-ae41-dda9aff83399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:55.118331 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f340d-d344-4451-ae41-dda9aff83399 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:55.118619 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f340d-d344-4451-ae41-dda9aff83399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:55.118902 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f340d-d344-4451-ae41-dda9aff83399 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:55.119914 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f340d-d344-4451-ae41-dda9aff83399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:55.122318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f340d-d344-4451-ae41-dda9aff83399 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:55.124390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e7cbf-2825-4ff0-af34-dd777b6d9504 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:55.129407 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1059dea-9f1a-4d36-9fad-8f91df9e6087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:55.130070 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1059dea-9f1a-4d36-9fad-8f91df9e6087 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:55.130362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1059dea-9f1a-4d36-9fad-8f91df9e6087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:55.130642 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1059dea-9f1a-4d36-9fad-8f91df9e6087 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:55.131648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1059dea-9f1a-4d36-9fad-8f91df9e6087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:55.133487 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1059dea-9f1a-4d36-9fad-8f91df9e6087 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:55.133805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 800/1593] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:55.135090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfcdf7-6a64-4279-8700-dda245d7a61a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:55.135383 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfcdf7-6a64-4279-8700-dda245d7a61a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:55.135692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfcdf7-6a64-4279-8700-dda245d7a61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:55.148202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 794/1594] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:55.148202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1059dea-9f1a-4d36-9fad-8f91df9e6087 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:55.148490 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d1059dea-9f1a-4d36-9fad-8f91df9e6087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:55.154413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-02dfcdf7-6a64-4279-8700-dda245d7a61a None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:55.156561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-02dfcdf7-6a64-4279-8700-dda245d7a61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:55.164553 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 801/1595] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:55.165369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dfcdf7-6a64-4279-8700-dda245d7a61a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:55.175142 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dfcdf7-6a64-4279-8700-dda245d7a61a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:55.180743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65472c-afa8-4d3a-b36f-9134a9e8dcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:55.181421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65472c-afa8-4d3a-b36f-9134a9e8dcf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:55.181708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65472c-afa8-4d3a-b36f-9134a9e8dcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:55.181988 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65472c-afa8-4d3a-b36f-9134a9e8dcf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:55.182726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65472c-afa8-4d3a-b36f-9134a9e8dcf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:55.184667 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65472c-afa8-4d3a-b36f-9134a9e8dcf7 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:55.195248 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 802/1596] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:55.196124 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65472c-afa8-4d3a-b36f-9134a9e8dcf7 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:55.207621 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65472c-afa8-4d3a-b36f-9134a9e8dcf7 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:55.208487 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:45:55.212641 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b89e5-80f9-4567-925c-68067237d2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:55.213293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b89e5-80f9-4567-925c-68067237d2e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:55.213583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b89e5-80f9-4567-925c-68067237d2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:55.213831 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1059dea-9f1a-4d36-9fad-8f91df9e6087 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:55.213970 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b89e5-80f9-4567-925c-68067237d2e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:55.214583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b89e5-80f9-4567-925c-68067237d2e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:55.216230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b89e5-80f9-4567-925c-68067237d2e7 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:55.218102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfd4b5-b570-4998-a002-0bff40d14af8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:55.218752 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfd4b5-b570-4998-a002-0bff40d14af8 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:55.219036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfd4b5-b570-4998-a002-0bff40d14af8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:55.219315 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfd4b5-b570-4998-a002-0bff40d14af8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:55.230869 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 803/1597] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:55.230869 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b89e5-80f9-4567-925c-68067237d2e7 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:55.232735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfd4b5-b570-4998-a002-0bff40d14af8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:55.234454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfd4b5-b570-4998-a002-0bff40d14af8 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:55.238547 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b89e5-80f9-4567-925c-68067237d2e7 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:55.243760 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413876e5-bfc3-4bc4-9db7-96eb9db7ef11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:55.244421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413876e5-bfc3-4bc4-9db7-96eb9db7ef11 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:55.244703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413876e5-bfc3-4bc4-9db7-96eb9db7ef11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:55.245097 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 795/1598] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:45:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:45:55.245215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413876e5-bfc3-4bc4-9db7-96eb9db7ef11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:55.245791 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfd4b5-b570-4998-a002-0bff40d14af8 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:55.245899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413876e5-bfc3-4bc4-9db7-96eb9db7ef11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:55.248101 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413876e5-bfc3-4bc4-9db7-96eb9db7ef11 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:55.255375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfd4b5-b570-4998-a002-0bff40d14af8 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:55.266876 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c74b0-d04f-4dd2-ad13-f38a1318ba6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:55.267696 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c74b0-d04f-4dd2-ad13-f38a1318ba6f admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:55.268063 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c74b0-d04f-4dd2-ad13-f38a1318ba6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:55.268431 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c74b0-d04f-4dd2-ad13-f38a1318ba6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:55.269321 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c74b0-d04f-4dd2-ad13-f38a1318ba6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:55.270906 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c74b0-d04f-4dd2-ad13-f38a1318ba6f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:55.286670 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 796/1599] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:55.286670 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c74b0-d04f-4dd2-ad13-f38a1318ba6f admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:55.293385 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c74b0-d04f-4dd2-ad13-f38a1318ba6f admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:55.298400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2dc2a-e389-49f3-bbcc-bf32e91d1c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:55.299054 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2dc2a-e389-49f3-bbcc-bf32e91d1c21 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:55.299340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2dc2a-e389-49f3-bbcc-bf32e91d1c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:55.299621 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2dc2a-e389-49f3-bbcc-bf32e91d1c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:55.300383 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2dc2a-e389-49f3-bbcc-bf32e91d1c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:55.301979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2dc2a-e389-49f3-bbcc-bf32e91d1c21 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:55.312586 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 797/1600] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:55.317357 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2dc2a-e389-49f3-bbcc-bf32e91d1c21 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:55.323398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2dc2a-e389-49f3-bbcc-bf32e91d1c21 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:55.329288 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaef60f-9215-4a88-b034-63fd40b19e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:55.329953 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaef60f-9215-4a88-b034-63fd40b19e5d admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:55.330237 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaef60f-9215-4a88-b034-63fd40b19e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:55.330513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaef60f-9215-4a88-b034-63fd40b19e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:55.331242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaef60f-9215-4a88-b034-63fd40b19e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:55.332880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaef60f-9215-4a88-b034-63fd40b19e5d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:55.344634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 798/1601] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:55.345473 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaef60f-9215-4a88-b034-63fd40b19e5d admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:55.355208 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaef60f-9215-4a88-b034-63fd40b19e5d admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:55.358968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 804/1602] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:45:55 2022] POST /identity/v3/projects => generated 363 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:55.359197 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1049f-1e95-4014-95b6-c8502f54fe1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:55.359551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1049f-1e95-4014-95b6-c8502f54fe1f service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:55.359700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413876e5-bfc3-4bc4-9db7-96eb9db7ef11 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:55.359885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1049f-1e95-4014-95b6-c8502f54fe1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:55.360163 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1049f-1e95-4014-95b6-c8502f54fe1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:55.370579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d1049f-1e95-4014-95b6-c8502f54fe1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:55.372079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d1049f-1e95-4014-95b6-c8502f54fe1f service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:55.382644 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 799/1603] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:45:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:45:55.383795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1049f-1e95-4014-95b6-c8502f54fe1f service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:55.393944 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1049f-1e95-4014-95b6-c8502f54fe1f service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:55.419990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:45:55.455694 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413876e5-bfc3-4bc4-9db7-96eb9db7ef11 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:55.461497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e91c7e-3e87-4754-8100-66d8501cba0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:55.462433 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e91c7e-3e87-4754-8100-66d8501cba0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:55.462720 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e91c7e-3e87-4754-8100-66d8501cba0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:55.463249 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e91c7e-3e87-4754-8100-66d8501cba0c admin admin] PATH_INFO: `/v3/projects/324cdcf46dde4aa987a2907fb89712fa/users/1ee0885d9e78456a90a30b5aa245ea64/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:55.474676 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03101e59-1493-4f66-9fc2-08ef96db4ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:55.475423 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03101e59-1493-4f66-9fc2-08ef96db4ffd admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:55.475761 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03101e59-1493-4f66-9fc2-08ef96db4ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:55.476281 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03101e59-1493-4f66-9fc2-08ef96db4ffd admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:55.476974 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03101e59-1493-4f66-9fc2-08ef96db4ffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:55.478592 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03101e59-1493-4f66-9fc2-08ef96db4ffd admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:55.480548 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e91c7e-3e87-4754-8100-66d8501cba0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=324cdcf46dde4aa987a2907fb89712fa, user_id=1ee0885d9e78456a90a30b5aa245ea64, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:55.483656 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e91c7e-3e87-4754-8100-66d8501cba0c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:55.492625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 800/1604] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:55.492625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03101e59-1493-4f66-9fc2-08ef96db4ffd admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:55.502375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03101e59-1493-4f66-9fc2-08ef96db4ffd admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:55.507461 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667dd2c-937d-4f8e-8a16-565fb2620c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:55.508422 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667dd2c-937d-4f8e-8a16-565fb2620c00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:55.508709 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667dd2c-937d-4f8e-8a16-565fb2620c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:55.508992 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667dd2c-937d-4f8e-8a16-565fb2620c00 admin admin] PATH_INFO: `/v3/projects/ab95af0a9cb74221a15ac73cbb8ca0ff/users/4e6b788e0819483580c09902a341ea6d/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:55.527279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667dd2c-937d-4f8e-8a16-565fb2620c00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab95af0a9cb74221a15ac73cbb8ca0ff, user_id=4e6b788e0819483580c09902a341ea6d, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:55.530124 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667dd2c-937d-4f8e-8a16-565fb2620c00 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:55.613954 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 801/1605] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:45:55 2022] PUT /identity/v3/projects/ab95af0a9cb74221a15ac73cbb8ca0ff/users/4e6b788e0819483580c09902a341ea6d/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:45:55.615842 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667dd2c-937d-4f8e-8a16-565fb2620c00 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:55.622016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 805/1606] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:45:55 2022] PUT /identity/v3/projects/324cdcf46dde4aa987a2907fb89712fa/users/1ee0885d9e78456a90a30b5aa245ea64/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:45:55.622894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e91c7e-3e87-4754-8100-66d8501cba0c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:55.695843 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667dd2c-937d-4f8e-8a16-565fb2620c00 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:55.702900 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03bd8e-cbf6-4ce5-a4ed-60f26fb530ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:55.705920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03bd8e-cbf6-4ce5-a4ed-60f26fb530ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:55.705920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03bd8e-cbf6-4ce5-a4ed-60f26fb530ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:55.705920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03bd8e-cbf6-4ce5-a4ed-60f26fb530ab admin admin] PATH_INFO: `/v3/projects/054149338e784f89b80b16d5be943b2b/users/f8157c1286a24d23ba68f8b4bb72ad71/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:55.723607 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03bd8e-cbf6-4ce5-a4ed-60f26fb530ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=054149338e784f89b80b16d5be943b2b, user_id=f8157c1286a24d23ba68f8b4bb72ad71, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:55.728931 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03bd8e-cbf6-4ce5-a4ed-60f26fb530ab admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:55.742034 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e91c7e-3e87-4754-8100-66d8501cba0c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:55.747261 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e426279-3207-4b78-bb38-d5d1947ed494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:55.748237 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e426279-3207-4b78-bb38-d5d1947ed494 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:55.748856 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e426279-3207-4b78-bb38-d5d1947ed494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:55.749039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e426279-3207-4b78-bb38-d5d1947ed494 admin admin] PATH_INFO: `/v3/projects/c2f44b51cc624637bccd9ed1d3b1a769/users/56b3d8d145d94b358c86345eb5c3b306/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:55.764520 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e426279-3207-4b78-bb38-d5d1947ed494 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2f44b51cc624637bccd9ed1d3b1a769, user_id=56b3d8d145d94b358c86345eb5c3b306, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:55.767442 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e426279-3207-4b78-bb38-d5d1947ed494 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:55.833999 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:45:55.838779 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 806/1607] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:45:55 2022] PUT /identity/v3/projects/c2f44b51cc624637bccd9ed1d3b1a769/users/56b3d8d145d94b358c86345eb5c3b306/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:45:55.843574 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e426279-3207-4b78-bb38-d5d1947ed494 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:55.854052 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 802/1608] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:45:55 2022] PUT /identity/v3/projects/054149338e784f89b80b16d5be943b2b/users/f8157c1286a24d23ba68f8b4bb72ad71/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:45:55.855108 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03bd8e-cbf6-4ce5-a4ed-60f26fb530ab admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:55.941315 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03bd8e-cbf6-4ce5-a4ed-60f26fb530ab admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:55.945267 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e426279-3207-4b78-bb38-d5d1947ed494 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:55.946526 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7341b41-0ba0-4fcc-ac3c-fb0b7642ee14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:55.947584 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7341b41-0ba0-4fcc-ac3c-fb0b7642ee14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:55.948074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7341b41-0ba0-4fcc-ac3c-fb0b7642ee14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:55.948373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7341b41-0ba0-4fcc-ac3c-fb0b7642ee14 admin admin] PATH_INFO: `/v3/projects/d7367af701b84358a4e5c5d0db473b44/users/94eed85a730b46dcb1cf694a3ebf2d3a/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:55.953226 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1bb689-2d7a-4e91-9fc6-5d896f20bef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:55.954029 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bb689-2d7a-4e91-9fc6-5d896f20bef8 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:55.954416 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bb689-2d7a-4e91-9fc6-5d896f20bef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:55.954798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bb689-2d7a-4e91-9fc6-5d896f20bef8 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:55.955757 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1bb689-2d7a-4e91-9fc6-5d896f20bef8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:55.958382 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1bb689-2d7a-4e91-9fc6-5d896f20bef8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:55.969837 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7341b41-0ba0-4fcc-ac3c-fb0b7642ee14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7367af701b84358a4e5c5d0db473b44, user_id=94eed85a730b46dcb1cf694a3ebf2d3a, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:55.970342 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7341b41-0ba0-4fcc-ac3c-fb0b7642ee14 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:56.031923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 807/1609] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:45:55 2022] POST /identity/v3/users => generated 288 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:56.033968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0481019-0501-48c7-832f-6ce67927e092 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:56.034437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0481019-0501-48c7-832f-6ce67927e092 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:56.034872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0481019-0501-48c7-832f-6ce67927e092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:56.035925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 803/1610] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:45:55 2022] PUT /identity/v3/projects/d7367af701b84358a4e5c5d0db473b44/users/94eed85a730b46dcb1cf694a3ebf2d3a/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:45:56.037747 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51561f-4069-4578-8461-a7981460ef02 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:56.038160 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51561f-4069-4578-8461-a7981460ef02 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:56.038696 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51561f-4069-4578-8461-a7981460ef02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:56.055521 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c0481019-0501-48c7-832f-6ce67927e092 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:56.056819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c0481019-0501-48c7-832f-6ce67927e092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:56.060145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-dc51561f-4069-4578-8461-a7981460ef02 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:56.062833 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-dc51561f-4069-4578-8461-a7981460ef02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:56.086607 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:45:56.137845 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 804/1611] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:56.139738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c20df-9bda-41c1-a2fd-b2a86c16fa4d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:56.140040 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c20df-9bda-41c1-a2fd-b2a86c16fa4d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:56.140325 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c20df-9bda-41c1-a2fd-b2a86c16fa4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:56.161592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 808/1612] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:56.163300 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d92cf6-ea34-4f09-a867-3d6d066a8a32 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:56.163821 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d92cf6-ea34-4f09-a867-3d6d066a8a32 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:56.164261 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d92cf6-ea34-4f09-a867-3d6d066a8a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:56.189547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-818c20df-9bda-41c1-a2fd-b2a86c16fa4d None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:56.190960 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-818c20df-9bda-41c1-a2fd-b2a86c16fa4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:56.209697 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 805/1613] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:56.212292 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c20df-9bda-41c1-a2fd-b2a86c16fa4d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:56.213976 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-98d92cf6-ea34-4f09-a867-3d6d066a8a32 None None] MFA Rules not processed for user `f8157c1286a24d23ba68f8b4bb72ad71`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:56.215489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-98d92cf6-ea34-4f09-a867-3d6d066a8a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:56.232829 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c20df-9bda-41c1-a2fd-b2a86c16fa4d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:56.243468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f771743-5f8a-4aa3-b912-c663efb6f9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:56.244815 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f771743-5f8a-4aa3-b912-c663efb6f9a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:56.245099 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f771743-5f8a-4aa3-b912-c663efb6f9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:56.245552 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f771743-5f8a-4aa3-b912-c663efb6f9a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:56.246740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f771743-5f8a-4aa3-b912-c663efb6f9a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:56.250278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f771743-5f8a-4aa3-b912-c663efb6f9a4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:56.269228 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 806/1614] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:56.274237 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5e921-e335-4e58-9584-52ef10846871 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:56.274590 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5e921-e335-4e58-9584-52ef10846871 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:56.274829 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5e921-e335-4e58-9584-52ef10846871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:56.299387 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a6b5e921-e335-4e58-9584-52ef10846871 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:56.301037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a6b5e921-e335-4e58-9584-52ef10846871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:56.311918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 807/1615] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:56.311918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5e921-e335-4e58-9584-52ef10846871 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:56.354616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 809/1616] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:56 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:56.356631 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d92cf6-ea34-4f09-a867-3d6d066a8a32 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:56.390461 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5e921-e335-4e58-9584-52ef10846871 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:56.397498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f925b-157a-44ff-8e92-20a1fe345159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:56.398704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f925b-157a-44ff-8e92-20a1fe345159 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:56.399265 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f925b-157a-44ff-8e92-20a1fe345159 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:56.399812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f925b-157a-44ff-8e92-20a1fe345159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:56.411988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f925b-157a-44ff-8e92-20a1fe345159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:56.423960 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f925b-157a-44ff-8e92-20a1fe345159 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:56.423960 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:45:56.428562 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 808/1617] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:45:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:45:56.429771 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f925b-157a-44ff-8e92-20a1fe345159 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:56.433064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d92cf6-ea34-4f09-a867-3d6d066a8a32 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:56.438118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d84590-4727-4e7c-ab58-14b1840c19d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:56.438878 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d84590-4727-4e7c-ab58-14b1840c19d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:56.439234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d84590-4727-4e7c-ab58-14b1840c19d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:56.439822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f925b-157a-44ff-8e92-20a1fe345159 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:56.439972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d84590-4727-4e7c-ab58-14b1840c19d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:56.444143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc9f58-1a23-4d30-b706-e14443b265c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:56.444914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc9f58-1a23-4d30-b706-e14443b265c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:56.445382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc9f58-1a23-4d30-b706-e14443b265c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:56.445754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc9f58-1a23-4d30-b706-e14443b265c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:56.451096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d84590-4727-4e7c-ab58-14b1840c19d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:56.453837 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d84590-4727-4e7c-ab58-14b1840c19d1 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:56.458010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc9f58-1a23-4d30-b706-e14443b265c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:56.460186 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc9f58-1a23-4d30-b706-e14443b265c1 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:56.471443 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 810/1618] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:45:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:45:56.475164 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 809/1619] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:45:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:45:56.475164 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc9f58-1a23-4d30-b706-e14443b265c1 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:56.475565 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d84590-4727-4e7c-ab58-14b1840c19d1 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:56.483011 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc9f58-1a23-4d30-b706-e14443b265c1 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:56.487219 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01528e11-711b-46ad-b08b-1a22621a1a83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:56.488563 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01528e11-711b-46ad-b08b-1a22621a1a83 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:56.488851 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01528e11-711b-46ad-b08b-1a22621a1a83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:56.489197 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01528e11-711b-46ad-b08b-1a22621a1a83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:56.502594 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01528e11-711b-46ad-b08b-1a22621a1a83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:56.504160 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01528e11-711b-46ad-b08b-1a22621a1a83 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:56.507615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d84590-4727-4e7c-ab58-14b1840c19d1 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:56.519955 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 810/1620] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:45:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:45:56.520900 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01528e11-711b-46ad-b08b-1a22621a1a83 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:56.522255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425f2b1-6b85-4a2a-a2ec-546e0549aece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:56.524399 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425f2b1-6b85-4a2a-a2ec-546e0549aece admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:56.524934 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425f2b1-6b85-4a2a-a2ec-546e0549aece admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:56.533311 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01528e11-711b-46ad-b08b-1a22621a1a83 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:56.536858 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425f2b1-6b85-4a2a-a2ec-546e0549aece admin admin] PATH_INFO: `/v3/projects/f507254c7200433abbc276fb95fcbb61/users/c14e84a3885d439d941d40c67defb0b8/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:56.543967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae98c3-e410-47fd-a230-c1acc16d1d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:56.543967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae98c3-e410-47fd-a230-c1acc16d1d73 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:56.543967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae98c3-e410-47fd-a230-c1acc16d1d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:56.543967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae98c3-e410-47fd-a230-c1acc16d1d73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:56.544401 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae98c3-e410-47fd-a230-c1acc16d1d73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:56.546699 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae98c3-e410-47fd-a230-c1acc16d1d73 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:56.579071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425f2b1-6b85-4a2a-a2ec-546e0549aece admin admin] RBAC: Authorizing `identity:create_grant(project_id=f507254c7200433abbc276fb95fcbb61, user_id=c14e84a3885d439d941d40c67defb0b8, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:56.581445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425f2b1-6b85-4a2a-a2ec-546e0549aece admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:56.696632 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 811/1621] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:45:56 2022] PUT /identity/v3/projects/f507254c7200433abbc276fb95fcbb61/users/c14e84a3885d439d941d40c67defb0b8/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:45:56.698859 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 811/1622] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:45:56 2022] POST /identity/v3/projects => generated 365 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:56.700724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425f2b1-6b85-4a2a-a2ec-546e0549aece admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:56.702031 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae98c3-e410-47fd-a230-c1acc16d1d73 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:56.754352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:45:56.772655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae98c3-e410-47fd-a230-c1acc16d1d73 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:56.777698 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b259aa4-e378-4430-a782-3b0a30d02953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:56.778464 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b259aa4-e378-4430-a782-3b0a30d02953 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:56.778805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b259aa4-e378-4430-a782-3b0a30d02953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:56.779091 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b259aa4-e378-4430-a782-3b0a30d02953 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:56.780054 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b259aa4-e378-4430-a782-3b0a30d02953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:56.782253 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b259aa4-e378-4430-a782-3b0a30d02953 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:56.784940 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425f2b1-6b85-4a2a-a2ec-546e0549aece admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:56.793256 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933638d-9f27-4bdd-aabd-404d42c2f1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:56.798573 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933638d-9f27-4bdd-aabd-404d42c2f1da admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:56.798573 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933638d-9f27-4bdd-aabd-404d42c2f1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:56.798573 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933638d-9f27-4bdd-aabd-404d42c2f1da admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:56.798573 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933638d-9f27-4bdd-aabd-404d42c2f1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:56.799731 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933638d-9f27-4bdd-aabd-404d42c2f1da admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:56.814671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 812/1623] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:56.816102 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933638d-9f27-4bdd-aabd-404d42c2f1da admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:56.829611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933638d-9f27-4bdd-aabd-404d42c2f1da admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:56.842913 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1457ed-c8fd-428d-ba00-cba6e40885f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:56.844172 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1457ed-c8fd-428d-ba00-cba6e40885f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:56.844951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1457ed-c8fd-428d-ba00-cba6e40885f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:56.844951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1457ed-c8fd-428d-ba00-cba6e40885f5 admin admin] PATH_INFO: `/v3/projects/f507254c7200433abbc276fb95fcbb61/users/c14e84a3885d439d941d40c67defb0b8/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:56.850373 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1457ed-c8fd-428d-ba00-cba6e40885f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f507254c7200433abbc276fb95fcbb61, user_id=c14e84a3885d439d941d40c67defb0b8, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:56.853374 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1457ed-c8fd-428d-ba00-cba6e40885f5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:56.874915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 813/1624] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:45:56 2022] PUT /identity/v3/projects/f507254c7200433abbc276fb95fcbb61/users/c14e84a3885d439d941d40c67defb0b8/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:45:56.879624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be23f8e-4d47-4e48-86b6-ecab224402f4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:56.879624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be23f8e-4d47-4e48-86b6-ecab224402f4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:56.880153 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be23f8e-4d47-4e48-86b6-ecab224402f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:56.913170 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8be23f8e-4d47-4e48-86b6-ecab224402f4 None None] MFA Rules not processed for user `c14e84a3885d439d941d40c67defb0b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:56.914509 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8be23f8e-4d47-4e48-86b6-ecab224402f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:56.974812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 812/1625] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:45:56 2022] POST /identity/v3/users => generated 289 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:56.974812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b259aa4-e378-4430-a782-3b0a30d02953 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:56.993310 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b259aa4-e378-4430-a782-3b0a30d02953 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:57.050693 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 814/1626] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:56 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:45:57.053739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be23f8e-4d47-4e48-86b6-ecab224402f4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:57.116503 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f4d2a-5c93-40b6-8e13-e25ff4001b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:57.124194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f4d2a-5c93-40b6-8e13-e25ff4001b96 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:57.124194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f4d2a-5c93-40b6-8e13-e25ff4001b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:57.124194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f4d2a-5c93-40b6-8e13-e25ff4001b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:57.124194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f4d2a-5c93-40b6-8e13-e25ff4001b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:57.124194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f4d2a-5c93-40b6-8e13-e25ff4001b96 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:57.144200 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 813/1627] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:57.147556 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f4d2a-5c93-40b6-8e13-e25ff4001b96 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:57.150823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be23f8e-4d47-4e48-86b6-ecab224402f4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:57.157810 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79773f-280d-4bc4-b208-168ea30344c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:57.178010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79773f-280d-4bc4-b208-168ea30344c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:57.178010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79773f-280d-4bc4-b208-168ea30344c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:57.178010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79773f-280d-4bc4-b208-168ea30344c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:57.178320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f4d2a-5c93-40b6-8e13-e25ff4001b96 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:57.178320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1a5d5-5e09-4f8b-9d12-90c74d738e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:57.178320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1a5d5-5e09-4f8b-9d12-90c74d738e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:57.178320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1a5d5-5e09-4f8b-9d12-90c74d738e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:57.178639 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1a5d5-5e09-4f8b-9d12-90c74d738e74 admin admin] PATH_INFO: `/v3/projects/900dafb4ca4d46369bbec6ddcc49dd9e/users/4f2be008a7724c3aa2cee5ee9073be9b/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:57.178728 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79773f-280d-4bc4-b208-168ea30344c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:57.178728 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79773f-280d-4bc4-b208-168ea30344c3 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:57.187587 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1a5d5-5e09-4f8b-9d12-90c74d738e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=900dafb4ca4d46369bbec6ddcc49dd9e, user_id=4f2be008a7724c3aa2cee5ee9073be9b, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:57.191037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1a5d5-5e09-4f8b-9d12-90c74d738e74 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:57.205975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 815/1628] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:45:57 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:45:57.294845 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 814/1629] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:45:57 2022] PUT /identity/v3/projects/900dafb4ca4d46369bbec6ddcc49dd9e/users/4f2be008a7724c3aa2cee5ee9073be9b/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:45:57.297769 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79773f-280d-4bc4-b208-168ea30344c3 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:57.344020 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79773f-280d-4bc4-b208-168ea30344c3 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:57.412629 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f28c1-4679-4b8b-b51c-3e15d30fe960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:57.413309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f28c1-4679-4b8b-b51c-3e15d30fe960 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:57.413595 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f28c1-4679-4b8b-b51c-3e15d30fe960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:57.413871 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f28c1-4679-4b8b-b51c-3e15d30fe960 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:57.414615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f28c1-4679-4b8b-b51c-3e15d30fe960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:57.416894 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f28c1-4679-4b8b-b51c-3e15d30fe960 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:57.428366 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 816/1630] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:57.432289 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1a5d5-5e09-4f8b-9d12-90c74d738e74 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:57.457004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1a5d5-5e09-4f8b-9d12-90c74d738e74 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:57.476829 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b978f-24d4-4ba1-864b-dce36c3b881d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:57.477760 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b978f-24d4-4ba1-864b-dce36c3b881d admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:57.478046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b978f-24d4-4ba1-864b-dce36c3b881d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:57.478325 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b978f-24d4-4ba1-864b-dce36c3b881d admin admin] PATH_INFO: `/v3/projects/900dafb4ca4d46369bbec6ddcc49dd9e/users/4f2be008a7724c3aa2cee5ee9073be9b/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:57.480965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b978f-24d4-4ba1-864b-dce36c3b881d admin admin] RBAC: Authorizing `identity:create_grant(project_id=900dafb4ca4d46369bbec6ddcc49dd9e, user_id=4f2be008a7724c3aa2cee5ee9073be9b, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:57.483897 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b978f-24d4-4ba1-864b-dce36c3b881d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:57.488667 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f28c1-4679-4b8b-b51c-3e15d30fe960 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:57.511347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:45:57.512174 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:45:57.524880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 815/1631] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:45:57 2022] PUT /identity/v3/projects/900dafb4ca4d46369bbec6ddcc49dd9e/users/4f2be008a7724c3aa2cee5ee9073be9b/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:45:57.538014 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa6a98-9a26-4f0a-af02-b0b79c979595 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:57.538782 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa6a98-9a26-4f0a-af02-b0b79c979595 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:57.539353 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa6a98-9a26-4f0a-af02-b0b79c979595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:57.606286 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f28c1-4679-4b8b-b51c-3e15d30fe960 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:57.649332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-f3fa6a98-9a26-4f0a-af02-b0b79c979595 None None] MFA Rules not processed for user `4f2be008a7724c3aa2cee5ee9073be9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:57.654299 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f3fa6a98-9a26-4f0a-af02-b0b79c979595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:57.724096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b72f7-56dc-419b-9389-0739a74e56bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:57.724775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b72f7-56dc-419b-9389-0739a74e56bc service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:57.725061 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b72f7-56dc-419b-9389-0739a74e56bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:57.725341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b72f7-56dc-419b-9389-0739a74e56bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:57.767253 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 816/1632] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:57 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:45:57.785341 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa6a98-9a26-4f0a-af02-b0b79c979595 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:57.817361 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa6a98-9a26-4f0a-af02-b0b79c979595 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:57.987469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ff01e-228d-4417-b7c4-c6666e63a386 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:57.987469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ff01e-228d-4417-b7c4-c6666e63a386 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:57.987469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ff01e-228d-4417-b7c4-c6666e63a386 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:57.987469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ff01e-228d-4417-b7c4-c6666e63a386 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:57.991989 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b72f7-56dc-419b-9389-0739a74e56bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:57.991989 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b72f7-56dc-419b-9389-0739a74e56bc service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:57.992977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ff01e-228d-4417-b7c4-c6666e63a386 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:57.992977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ff01e-228d-4417-b7c4-c6666e63a386 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:57.992977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 817/1633] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:45:57 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:45:57.993540 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 817/1634] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:45:57 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:45:58.992064 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e044e3-c53c-4bdc-9184-bbf2d474b5f1 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:58.992962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e044e3-c53c-4bdc-9184-bbf2d474b5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:58.993405 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e044e3-c53c-4bdc-9184-bbf2d474b5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:59.059404 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-e3e044e3-c53c-4bdc-9184-bbf2d474b5f1 None None] MFA Rules not processed for user `1ee0885d9e78456a90a30b5aa245ea64`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:45:59.061526 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e3e044e3-c53c-4bdc-9184-bbf2d474b5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:59.225910 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 818/1635] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:45:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:45:59.230502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b72f7-56dc-419b-9389-0739a74e56bc service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:59.366125 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:45:59.377553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e044e3-c53c-4bdc-9184-bbf2d474b5f1 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:59.412912 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b72f7-56dc-419b-9389-0739a74e56bc service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:59.416316 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc1d91-d8b2-4d59-9da1-c9950a661592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:59.418244 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc1d91-d8b2-4d59-9da1-c9950a661592 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:59.419877 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc1d91-d8b2-4d59-9da1-c9950a661592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:59.421238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc1d91-d8b2-4d59-9da1-c9950a661592 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:59.427883 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc1d91-d8b2-4d59-9da1-c9950a661592 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:59.432811 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc1d91-d8b2-4d59-9da1-c9950a661592 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:59.504256 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e044e3-c53c-4bdc-9184-bbf2d474b5f1 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:59.510911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b3d78-d410-479f-a723-28ee4d2f3b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:59.511609 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b3d78-d410-479f-a723-28ee4d2f3b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:59.512448 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b3d78-d410-479f-a723-28ee4d2f3b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:59.512747 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b3d78-d410-479f-a723-28ee4d2f3b35 admin admin] PATH_INFO: `/v3/users/f8157c1286a24d23ba68f8b4bb72ad71` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:59.515441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b3d78-d410-479f-a723-28ee4d2f3b35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8157c1286a24d23ba68f8b4bb72ad71)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:59.516706 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: WARNING py.warnings [None req-297b3d78-d410-479f-a723-28ee4d2f3b35 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 08:45:59.516706 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: warnings.warn(msg) Jul 18 08:45:59.516706 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: Jul 18 08:45:59.518019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b3d78-d410-479f-a723-28ee4d2f3b35 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:59.521812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:45:59.534817 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 818/1636] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:45:59 2022] POST /identity/v3/projects => generated 403 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:45:59.538617 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc1d91-d8b2-4d59-9da1-c9950a661592 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:59.679460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc1d91-d8b2-4d59-9da1-c9950a661592 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:59.687062 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b3b59-c114-4a1c-802f-21c904f145c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:59.688034 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b3b59-c114-4a1c-802f-21c904f145c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:59.688455 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b3b59-c114-4a1c-802f-21c904f145c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:59.689488 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b3b59-c114-4a1c-802f-21c904f145c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:59.690662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b3b59-c114-4a1c-802f-21c904f145c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:59.693338 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b3b59-c114-4a1c-802f-21c904f145c0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:59.804304 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 819/1637] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:45:59 2022] POST /identity/v3/users => generated 308 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:45:59.810107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-297b3d78-d410-479f-a723-28ee4d2f3b35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8157c1286a24d23ba68f8b4bb72ad71'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:45:59.810488 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b3b59-c114-4a1c-802f-21c904f145c0 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:59.811230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-297b3d78-d410-479f-a723-28ee4d2f3b35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8157c1286a24d23ba68f8b4bb72ad71'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:45:59.814418 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-297b3d78-d410-479f-a723-28ee4d2f3b35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8157c1286a24d23ba68f8b4bb72ad71'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:45:59.815621 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6c1b3b59-c114-4a1c-802f-21c904f145c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:45:59.847040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b3b59-c114-4a1c-802f-21c904f145c0 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:59.858891 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-297b3d78-d410-479f-a723-28ee4d2f3b35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8157c1286a24d23ba68f8b4bb72ad71'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:45:59.869981 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380480e-b7e5-4674-837c-eea4602b163c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:59.870694 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380480e-b7e5-4674-837c-eea4602b163c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:59.871237 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380480e-b7e5-4674-837c-eea4602b163c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:59.871529 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380480e-b7e5-4674-837c-eea4602b163c admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:45:59.872487 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380480e-b7e5-4674-837c-eea4602b163c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:45:59.876438 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380480e-b7e5-4674-837c-eea4602b163c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:45:59.919809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-297b3d78-d410-479f-a723-28ee4d2f3b35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8157c1286a24d23ba68f8b4bb72ad71'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:45:59.937730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 820/1638] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:45:59.939009 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380480e-b7e5-4674-837c-eea4602b163c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:45:59.970608 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380480e-b7e5-4674-837c-eea4602b163c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:45:59.985083 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248f38e-fbb4-40a8-b5e2-86210c836165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:45:59.986345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248f38e-fbb4-40a8-b5e2-86210c836165 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:45:59.986833 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248f38e-fbb4-40a8-b5e2-86210c836165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:45:59.987337 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248f38e-fbb4-40a8-b5e2-86210c836165 admin admin] PATH_INFO: `/v3/projects/bb330e7e37f2447a9ec98db9dd4c5a9e/users/b43d349d945649a396292ec7033af9dd/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:00.007975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248f38e-fbb4-40a8-b5e2-86210c836165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb330e7e37f2447a9ec98db9dd4c5a9e, user_id=b43d349d945649a396292ec7033af9dd, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:00.011587 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248f38e-fbb4-40a8-b5e2-86210c836165 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:00.065015 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 819/1639] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:45:59 2022] DELETE /identity/v3/users/f8157c1286a24d23ba68f8b4bb72ad71 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:00.070408 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82134b1-e643-416c-8ca6-b434b6847912 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:00.071813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82134b1-e643-416c-8ca6-b434b6847912 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:00.073932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82134b1-e643-416c-8ca6-b434b6847912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:00.109072 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 821/1640] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:45:59 2022] PUT /identity/v3/projects/bb330e7e37f2447a9ec98db9dd4c5a9e/users/b43d349d945649a396292ec7033af9dd/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:00.114741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248f38e-fbb4-40a8-b5e2-86210c836165 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:00.128280 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c82134b1-e643-416c-8ca6-b434b6847912 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:00.143881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c82134b1-e643-416c-8ca6-b434b6847912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:00.274062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 820/1641] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:00.297477 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248f38e-fbb4-40a8-b5e2-86210c836165 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:00.305832 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363bf94-37e3-4981-978e-1aab0f5ec160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:00.306671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363bf94-37e3-4981-978e-1aab0f5ec160 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:00.308161 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363bf94-37e3-4981-978e-1aab0f5ec160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:00.308161 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363bf94-37e3-4981-978e-1aab0f5ec160 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:00.308161 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363bf94-37e3-4981-978e-1aab0f5ec160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:00.335998 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363bf94-37e3-4981-978e-1aab0f5ec160 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:00.343157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82134b1-e643-416c-8ca6-b434b6847912 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:00.344843 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c82134b1-e643-416c-8ca6-b434b6847912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:00.387030 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 822/1642] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:00.388057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363bf94-37e3-4981-978e-1aab0f5ec160 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:00.427870 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363bf94-37e3-4981-978e-1aab0f5ec160 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:00.453270 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a75351-dcae-4298-8a9e-69473f0990bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:00.454247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a75351-dcae-4298-8a9e-69473f0990bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:00.454532 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a75351-dcae-4298-8a9e-69473f0990bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:00.454816 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a75351-dcae-4298-8a9e-69473f0990bf admin admin] PATH_INFO: `/v3/projects/bb330e7e37f2447a9ec98db9dd4c5a9e/users/b43d349d945649a396292ec7033af9dd/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:00.458872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a75351-dcae-4298-8a9e-69473f0990bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb330e7e37f2447a9ec98db9dd4c5a9e, user_id=b43d349d945649a396292ec7033af9dd, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:00.486995 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a75351-dcae-4298-8a9e-69473f0990bf admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:00.514389 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 823/1643] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:46:00 2022] PUT /identity/v3/projects/bb330e7e37f2447a9ec98db9dd4c5a9e/users/b43d349d945649a396292ec7033af9dd/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:00.558683 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82134b1-e643-416c-8ca6-b434b6847912 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:00.621820 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:00.640800 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc5ca66-31f2-4ef4-b29b-cb060a3d4be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:00.641480 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5ca66-31f2-4ef4-b29b-cb060a3d4be7 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:00.641761 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5ca66-31f2-4ef4-b29b-cb060a3d4be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:00.642043 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5ca66-31f2-4ef4-b29b-cb060a3d4be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:00.726847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc5ca66-31f2-4ef4-b29b-cb060a3d4be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:00.728362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc5ca66-31f2-4ef4-b29b-cb060a3d4be7 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:00.742273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 821/1644] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:46:02.857045 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a75351-dcae-4298-8a9e-69473f0990bf admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:02.868608 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-92a75351-dcae-4298-8a9e-69473f0990bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:02.936863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a75351-dcae-4298-8a9e-69473f0990bf admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:02.948892 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47202803-44e3-4ea8-b24e-5519d6e2d64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:02.953054 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47202803-44e3-4ea8-b24e-5519d6e2d64a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:02.953977 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47202803-44e3-4ea8-b24e-5519d6e2d64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:02.954788 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47202803-44e3-4ea8-b24e-5519d6e2d64a admin admin] PATH_INFO: `/v3/users/91c81496c57c471ea3858579b3aa83dc` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:02.959661 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47202803-44e3-4ea8-b24e-5519d6e2d64a admin admin] RBAC: Authorizing `identity:delete_user(user_id=91c81496c57c471ea3858579b3aa83dc)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:02.964915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47202803-44e3-4ea8-b24e-5519d6e2d64a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:03.002877 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:03.070290 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc5ca66-31f2-4ef4-b29b-cb060a3d4be7 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:03.074326 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7cc5ca66-31f2-4ef4-b29b-cb060a3d4be7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:03.115037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc5ca66-31f2-4ef4-b29b-cb060a3d4be7 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:03.128608 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b866235-327e-4137-81a3-eb999ab17ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:03.129513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b866235-327e-4137-81a3-eb999ab17ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:03.129789 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b866235-327e-4137-81a3-eb999ab17ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:03.130074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b866235-327e-4137-81a3-eb999ab17ccc admin admin] PATH_INFO: `/v3/projects/054149338e784f89b80b16d5be943b2b` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:03.133067 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b866235-327e-4137-81a3-eb999ab17ccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=054149338e784f89b80b16d5be943b2b)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:03.139445 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b866235-327e-4137-81a3-eb999ab17ccc admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:03.335584 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-47202803-44e3-4ea8-b24e-5519d6e2d64a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91c81496c57c471ea3858579b3aa83dc'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:03.336963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-47202803-44e3-4ea8-b24e-5519d6e2d64a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91c81496c57c471ea3858579b3aa83dc'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:03.337779 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-47202803-44e3-4ea8-b24e-5519d6e2d64a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91c81496c57c471ea3858579b3aa83dc'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:03.385638 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-47202803-44e3-4ea8-b24e-5519d6e2d64a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91c81496c57c471ea3858579b3aa83dc'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:03.387980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0b866235-327e-4137-81a3-eb999ab17ccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '054149338e784f89b80b16d5be943b2b', 'request_id': 'req-0b866235-327e-4137-81a3-eb999ab17ccc'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:03.406311 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0b866235-327e-4137-81a3-eb999ab17ccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '054149338e784f89b80b16d5be943b2b', 'request_id': 'req-0b866235-327e-4137-81a3-eb999ab17ccc'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:03.408867 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-47202803-44e3-4ea8-b24e-5519d6e2d64a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91c81496c57c471ea3858579b3aa83dc'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:03.425983 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0b866235-327e-4137-81a3-eb999ab17ccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '054149338e784f89b80b16d5be943b2b', 'request_id': 'req-0b866235-327e-4137-81a3-eb999ab17ccc'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:03.529448 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 824/1645] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:46:02 2022] DELETE /identity/v3/users/91c81496c57c471ea3858579b3aa83dc => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:03.536303 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f901e-8c2b-4e11-866f-577b835516dd None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:03.536676 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f901e-8c2b-4e11-866f-577b835516dd None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:03.537185 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f901e-8c2b-4e11-866f-577b835516dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:03.580813 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-df6f901e-8c2b-4e11-866f-577b835516dd None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:03.584290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-df6f901e-8c2b-4e11-866f-577b835516dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:03.619440 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0b866235-327e-4137-81a3-eb999ab17ccc admin admin] The token cache is being invalidate because project 054149338e784f89b80b16d5be943b2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:46:03.619828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0b866235-327e-4137-81a3-eb999ab17ccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:03.620804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0b866235-327e-4137-81a3-eb999ab17ccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:03.631617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 822/1646] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:46:03 2022] DELETE /identity/v3/projects/054149338e784f89b80b16d5be943b2b => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:03.631617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b866235-327e-4137-81a3-eb999ab17ccc admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:03.640946 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0b866235-327e-4137-81a3-eb999ab17ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:03.769716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 825/1647] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:46:03.821706 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b866235-327e-4137-81a3-eb999ab17ccc admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:03.879969 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6378a5-1176-4bf3-b29b-f067afbd08e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:03.882403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6378a5-1176-4bf3-b29b-f067afbd08e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:03.883765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6378a5-1176-4bf3-b29b-f067afbd08e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:03.884040 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6378a5-1176-4bf3-b29b-f067afbd08e3 admin admin] PATH_INFO: `/v3/users/4f2be008a7724c3aa2cee5ee9073be9b` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:03.892194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6378a5-1176-4bf3-b29b-f067afbd08e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f2be008a7724c3aa2cee5ee9073be9b)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:03.895985 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f901e-8c2b-4e11-866f-577b835516dd None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:03.900355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-df6f901e-8c2b-4e11-866f-577b835516dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:03.900899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6378a5-1176-4bf3-b29b-f067afbd08e3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:03.985777 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:04.163068 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f901e-8c2b-4e11-866f-577b835516dd None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:04.170813 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e97e0-ea59-4022-b0c9-d6327a9e3615 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:04.171842 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e97e0-ea59-4022-b0c9-d6327a9e3615 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:04.172663 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e97e0-ea59-4022-b0c9-d6327a9e3615 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:04.173103 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e97e0-ea59-4022-b0c9-d6327a9e3615 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:04.212287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e97e0-ea59-4022-b0c9-d6327a9e3615 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:04.214188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e97e0-ea59-4022-b0c9-d6327a9e3615 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:04.284156 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 826/1648] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:46:04.340219 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5d6378a5-1176-4bf3-b29b-f067afbd08e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f2be008a7724c3aa2cee5ee9073be9b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:04.343241 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5d6378a5-1176-4bf3-b29b-f067afbd08e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f2be008a7724c3aa2cee5ee9073be9b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:04.344770 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5d6378a5-1176-4bf3-b29b-f067afbd08e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f2be008a7724c3aa2cee5ee9073be9b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:04.388311 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5d6378a5-1176-4bf3-b29b-f067afbd08e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f2be008a7724c3aa2cee5ee9073be9b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:04.408625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5d6378a5-1176-4bf3-b29b-f067afbd08e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f2be008a7724c3aa2cee5ee9073be9b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:04.503103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 823/1649] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:46:03 2022] DELETE /identity/v3/users/4f2be008a7724c3aa2cee5ee9073be9b => generated 0 bytes in 874 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:04.916307 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e74c3-4e67-46b6-ad9f-28979f8f0dc5 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:04.916836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e74c3-4e67-46b6-ad9f-28979f8f0dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:04.916940 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e74c3-4e67-46b6-ad9f-28979f8f0dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:04.971990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-121e74c3-4e67-46b6-ad9f-28979f8f0dc5 None None] MFA Rules not processed for user `b43d349d945649a396292ec7033af9dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:04.973969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-121e74c3-4e67-46b6-ad9f-28979f8f0dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:05.016306 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:05.156145 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 827/1650] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:04 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:06.265232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6378a5-1176-4bf3-b29b-f067afbd08e3 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:06.273903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5d6378a5-1176-4bf3-b29b-f067afbd08e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:06.571989 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6378a5-1176-4bf3-b29b-f067afbd08e3 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:06.597643 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae38d34-d96d-4824-9ab0-58eed8901471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:06.598707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae38d34-d96d-4824-9ab0-58eed8901471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:06.599374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae38d34-d96d-4824-9ab0-58eed8901471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:06.606326 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae38d34-d96d-4824-9ab0-58eed8901471 admin admin] PATH_INFO: `/v3/projects/900dafb4ca4d46369bbec6ddcc49dd9e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:06.606326 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae38d34-d96d-4824-9ab0-58eed8901471 admin admin] RBAC: Authorizing `identity:delete_project(project_id=900dafb4ca4d46369bbec6ddcc49dd9e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:06.623173 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae38d34-d96d-4824-9ab0-58eed8901471 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:06.936800 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9ae38d34-d96d-4824-9ab0-58eed8901471 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '900dafb4ca4d46369bbec6ddcc49dd9e', 'request_id': 'req-9ae38d34-d96d-4824-9ab0-58eed8901471'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:06.975929 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9ae38d34-d96d-4824-9ab0-58eed8901471 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '900dafb4ca4d46369bbec6ddcc49dd9e', 'request_id': 'req-9ae38d34-d96d-4824-9ab0-58eed8901471'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:06.993753 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9ae38d34-d96d-4824-9ab0-58eed8901471 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '900dafb4ca4d46369bbec6ddcc49dd9e', 'request_id': 'req-9ae38d34-d96d-4824-9ab0-58eed8901471'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:07.101882 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9ae38d34-d96d-4824-9ab0-58eed8901471 admin admin] The token cache is being invalidate because project 900dafb4ca4d46369bbec6ddcc49dd9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:46:07.102493 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9ae38d34-d96d-4824-9ab0-58eed8901471 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:07.104226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9ae38d34-d96d-4824-9ab0-58eed8901471 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:07.106722 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 824/1651] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:46:06 2022] DELETE /identity/v3/projects/900dafb4ca4d46369bbec6ddcc49dd9e => generated 0 bytes in 849 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:07.114987 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0a94f-63c4-460c-afd3-f855c89af796 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:07.115899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0a94f-63c4-460c-afd3-f855c89af796 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:07.116251 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0a94f-63c4-460c-afd3-f855c89af796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:07.179153 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5ec0a94f-63c4-460c-afd3-f855c89af796 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:07.180969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5ec0a94f-63c4-460c-afd3-f855c89af796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:07.375120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 828/1652] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:46:07.390907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a45b4-ed68-4794-b53c-26f1b135d6be None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:07.391922 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a45b4-ed68-4794-b53c-26f1b135d6be None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:07.391922 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a45b4-ed68-4794-b53c-26f1b135d6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:07.440597 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-343a45b4-ed68-4794-b53c-26f1b135d6be None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:07.442722 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-343a45b4-ed68-4794-b53c-26f1b135d6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:07.456109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 825/1653] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:07.464986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0a94f-63c4-460c-afd3-f855c89af796 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:07.485234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0a94f-63c4-460c-afd3-f855c89af796 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:07.501184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337734a-368e-4d29-b53c-076ec8c2de8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:07.501898 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337734a-368e-4d29-b53c-076ec8c2de8a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:07.502190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337734a-368e-4d29-b53c-076ec8c2de8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:07.505688 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337734a-368e-4d29-b53c-076ec8c2de8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:07.508300 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337734a-368e-4d29-b53c-076ec8c2de8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:46:07.508433 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337734a-368e-4d29-b53c-076ec8c2de8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:07.511712 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337734a-368e-4d29-b53c-076ec8c2de8a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:07.590881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a45b4-ed68-4794-b53c-26f1b135d6be None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:07.606259 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 829/1654] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:46:07.618284 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a45b4-ed68-4794-b53c-26f1b135d6be None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:07.633773 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde063c7-e872-418b-b510-a920c95a4636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:07.634359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde063c7-e872-418b-b510-a920c95a4636 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:07.634633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde063c7-e872-418b-b510-a920c95a4636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:07.634912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde063c7-e872-418b-b510-a920c95a4636 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:07.635629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde063c7-e872-418b-b510-a920c95a4636 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:07.650752 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde063c7-e872-418b-b510-a920c95a4636 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:07.691213 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:07.798771 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 826/1655] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:46:07 2022] POST /identity/v3/projects => generated 357 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:46:07.804817 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337734a-368e-4d29-b53c-076ec8c2de8a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:07.821431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337734a-368e-4d29-b53c-076ec8c2de8a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:08.049281 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4f508-af57-4e76-9ca2-0daf870b752c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:08.050066 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4f508-af57-4e76-9ca2-0daf870b752c admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:08.050358 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4f508-af57-4e76-9ca2-0daf870b752c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:08.050701 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4f508-af57-4e76-9ca2-0daf870b752c admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:08.051614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4f508-af57-4e76-9ca2-0daf870b752c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:08.055007 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4f508-af57-4e76-9ca2-0daf870b752c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:08.179636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 830/1656] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:46:07 2022] POST /identity/v3/users => generated 279 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:46:08.184212 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde063c7-e872-418b-b510-a920c95a4636 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:08.204830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde063c7-e872-418b-b510-a920c95a4636 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:08.222583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e918b7a-e78e-4812-8ca6-f289e28f7f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:08.223947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e918b7a-e78e-4812-8ca6-f289e28f7f52 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:08.224383 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e918b7a-e78e-4812-8ca6-f289e28f7f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:08.224830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e918b7a-e78e-4812-8ca6-f289e28f7f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:08.228632 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e918b7a-e78e-4812-8ca6-f289e28f7f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:08.230728 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e918b7a-e78e-4812-8ca6-f289e28f7f52 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:08.270575 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 827/1657] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:08.275619 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4f508-af57-4e76-9ca2-0daf870b752c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:08.320582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4f508-af57-4e76-9ca2-0daf870b752c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:08.334557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67bf4c2-025d-45d6-907b-2eba02c3844e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:08.338672 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bf4c2-025d-45d6-907b-2eba02c3844e admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:08.338672 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bf4c2-025d-45d6-907b-2eba02c3844e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:08.338672 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bf4c2-025d-45d6-907b-2eba02c3844e admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:08.342766 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67bf4c2-025d-45d6-907b-2eba02c3844e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:08.347357 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67bf4c2-025d-45d6-907b-2eba02c3844e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:08.377907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 831/1658] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:08.383505 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e918b7a-e78e-4812-8ca6-f289e28f7f52 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:08.418257 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e918b7a-e78e-4812-8ca6-f289e28f7f52 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:08.460271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bc643-aaa4-4d8d-9310-c53b46daa167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:08.462492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bc643-aaa4-4d8d-9310-c53b46daa167 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:08.463002 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bc643-aaa4-4d8d-9310-c53b46daa167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:08.463467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bc643-aaa4-4d8d-9310-c53b46daa167 admin admin] PATH_INFO: `/v3/projects/48abe55177574cdb89c1e5725afc888b/users/72cac493db1642acb9e367e045624f03/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:08.515577 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45bc643-aaa4-4d8d-9310-c53b46daa167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48abe55177574cdb89c1e5725afc888b, user_id=72cac493db1642acb9e367e045624f03, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:08.533680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45bc643-aaa4-4d8d-9310-c53b46daa167 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:08.664303 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 828/1659] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:46:08 2022] PUT /identity/v3/projects/48abe55177574cdb89c1e5725afc888b/users/72cac493db1642acb9e367e045624f03/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:08.677302 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac4b05-3f29-464c-b8e3-d329d6f1c5e5 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:08.677831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac4b05-3f29-464c-b8e3-d329d6f1c5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:08.678353 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac4b05-3f29-464c-b8e3-d329d6f1c5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:08.791461 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a1ac4b05-3f29-464c-b8e3-d329d6f1c5e5 None None] MFA Rules not processed for user `72cac493db1642acb9e367e045624f03`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:08.797383 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a1ac4b05-3f29-464c-b8e3-d329d6f1c5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:08.840560 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:09.063868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bc643-aaa4-4d8d-9310-c53b46daa167 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:09.064921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 832/1660] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:08 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:09.109310 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac4b05-3f29-464c-b8e3-d329d6f1c5e5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:09.111416 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a1ac4b05-3f29-464c-b8e3-d329d6f1c5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:09.260477 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac4b05-3f29-464c-b8e3-d329d6f1c5e5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:09.268040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b49695-747f-4ee6-b816-28e60e717f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:09.269258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b49695-747f-4ee6-b816-28e60e717f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:09.269697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b49695-747f-4ee6-b816-28e60e717f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:09.270116 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b49695-747f-4ee6-b816-28e60e717f76 admin admin] PATH_INFO: `/v3/projects/a5746e9b1fe24e929b4d3355b6605423` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:09.271657 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b49695-747f-4ee6-b816-28e60e717f76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5746e9b1fe24e929b4d3355b6605423)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:09.272764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: WARNING py.warnings [None req-b0b49695-747f-4ee6-b816-28e60e717f76 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 08:46:09.272764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: warnings.warn(msg) Jul 18 08:46:09.272764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: Jul 18 08:46:09.274520 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b49695-747f-4ee6-b816-28e60e717f76 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:09.383128 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bc643-aaa4-4d8d-9310-c53b46daa167 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:09.390069 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd14b1d-293e-4954-854d-d5e028cd9ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:09.392024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd14b1d-293e-4954-854d-d5e028cd9ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:09.392537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd14b1d-293e-4954-854d-d5e028cd9ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:09.392995 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd14b1d-293e-4954-854d-d5e028cd9ea1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:09.394185 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd14b1d-293e-4954-854d-d5e028cd9ea1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:09.398165 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd14b1d-293e-4954-854d-d5e028cd9ea1 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:09.492995 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b0b49695-747f-4ee6-b816-28e60e717f76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5746e9b1fe24e929b4d3355b6605423', 'request_id': 'req-b0b49695-747f-4ee6-b816-28e60e717f76'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:09.517884 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b0b49695-747f-4ee6-b816-28e60e717f76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5746e9b1fe24e929b4d3355b6605423', 'request_id': 'req-b0b49695-747f-4ee6-b816-28e60e717f76'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:09.561538 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b0b49695-747f-4ee6-b816-28e60e717f76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5746e9b1fe24e929b4d3355b6605423', 'request_id': 'req-b0b49695-747f-4ee6-b816-28e60e717f76'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:09.573871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 829/1661] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:46:09 2022] POST /identity/v3/projects => generated 357 bytes in 511 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:46:09.588700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd14b1d-293e-4954-854d-d5e028cd9ea1 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:09.771134 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd14b1d-293e-4954-854d-d5e028cd9ea1 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:09.785136 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c09da-8d44-440d-9387-14fa9ebb65e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:09.785953 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c09da-8d44-440d-9387-14fa9ebb65e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:09.787085 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c09da-8d44-440d-9387-14fa9ebb65e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:09.787085 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c09da-8d44-440d-9387-14fa9ebb65e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:09.787322 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c09da-8d44-440d-9387-14fa9ebb65e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:09.792867 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c09da-8d44-440d-9387-14fa9ebb65e4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:09.872226 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b0b49695-747f-4ee6-b816-28e60e717f76 admin admin] The token cache is being invalidate because project a5746e9b1fe24e929b4d3355b6605423 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:46:09.872226 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b0b49695-747f-4ee6-b816-28e60e717f76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:09.882211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b0b49695-747f-4ee6-b816-28e60e717f76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:09.885395 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 833/1662] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:46:09 2022] DELETE /identity/v3/projects/a5746e9b1fe24e929b4d3355b6605423 => generated 0 bytes in 776 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:09.890323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35210f11-9fd6-42df-b299-c6219cebd975 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:09.891280 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35210f11-9fd6-42df-b299-c6219cebd975 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:09.891280 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35210f11-9fd6-42df-b299-c6219cebd975 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:09.895879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 834/1663] 10.222.0.57 () {54 vars in 921 bytes} [Mon Jul 18 08:46:09 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:46:09.900378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35210f11-9fd6-42df-b299-c6219cebd975 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:09.916942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-35210f11-9fd6-42df-b299-c6219cebd975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:09.972967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 830/1664] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:46:09 2022] POST /identity/v3/users => generated 285 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:46:09.977432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c09da-8d44-440d-9387-14fa9ebb65e4 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:09.978751 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35210f11-9fd6-42df-b299-c6219cebd975 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:09.981008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6d6c09da-8d44-440d-9387-14fa9ebb65e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:10.016765 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d40c1-853e-4fb0-b923-be7a4a358a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:10.018115 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d40c1-853e-4fb0-b923-be7a4a358a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:10.020730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d40c1-853e-4fb0-b923-be7a4a358a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:10.021380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d40c1-853e-4fb0-b923-be7a4a358a15 admin admin] PATH_INFO: `/v3/users/c14e84a3885d439d941d40c67defb0b8` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:10.025572 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d40c1-853e-4fb0-b923-be7a4a358a15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c14e84a3885d439d941d40c67defb0b8)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:10.032550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d40c1-853e-4fb0-b923-be7a4a358a15 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:10.034701 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c09da-8d44-440d-9387-14fa9ebb65e4 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:10.057874 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254854fd-b727-4b33-9f1c-4293808b2e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:10.062835 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254854fd-b727-4b33-9f1c-4293808b2e7e admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:10.063196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254854fd-b727-4b33-9f1c-4293808b2e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:10.063476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254854fd-b727-4b33-9f1c-4293808b2e7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:10.064320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254854fd-b727-4b33-9f1c-4293808b2e7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:10.068390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254854fd-b727-4b33-9f1c-4293808b2e7e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:10.099449 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 831/1665] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:46:10.103239 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254854fd-b727-4b33-9f1c-4293808b2e7e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:10.134071 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254854fd-b727-4b33-9f1c-4293808b2e7e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:10.159058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f412c9-3608-428e-8f2c-0315a0225449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:10.160110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f412c9-3608-428e-8f2c-0315a0225449 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:10.160431 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f412c9-3608-428e-8f2c-0315a0225449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:10.160745 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f412c9-3608-428e-8f2c-0315a0225449 admin admin] PATH_INFO: `/v3/projects/b2320c44ac1c4954ab40fb192e9a4254/users/0349a014e40f422bb1b5a6150f60cf60/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:10.197833 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f412c9-3608-428e-8f2c-0315a0225449 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2320c44ac1c4954ab40fb192e9a4254, user_id=0349a014e40f422bb1b5a6150f60cf60, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:10.203629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f412c9-3608-428e-8f2c-0315a0225449 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:10.215024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:10.312245 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 832/1666] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:46:10 2022] PUT /identity/v3/projects/b2320c44ac1c4954ab40fb192e9a4254/users/0349a014e40f422bb1b5a6150f60cf60/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:10.312245 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f412c9-3608-428e-8f2c-0315a0225449 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:10.396825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:10.414062 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-517d40c1-853e-4fb0-b923-be7a4a358a15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c14e84a3885d439d941d40c67defb0b8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:10.416143 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-517d40c1-853e-4fb0-b923-be7a4a358a15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c14e84a3885d439d941d40c67defb0b8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:10.417717 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-517d40c1-853e-4fb0-b923-be7a4a358a15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c14e84a3885d439d941d40c67defb0b8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:10.440416 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-517d40c1-853e-4fb0-b923-be7a4a358a15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c14e84a3885d439d941d40c67defb0b8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:10.452649 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-517d40c1-853e-4fb0-b923-be7a4a358a15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c14e84a3885d439d941d40c67defb0b8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:10.514212 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f412c9-3608-428e-8f2c-0315a0225449 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:10.538709 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94560af4-999a-4636-9fa5-72314280e7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:10.539796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94560af4-999a-4636-9fa5-72314280e7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:10.539796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94560af4-999a-4636-9fa5-72314280e7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:10.540051 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94560af4-999a-4636-9fa5-72314280e7b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:10.540731 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94560af4-999a-4636-9fa5-72314280e7b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:10.542387 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94560af4-999a-4636-9fa5-72314280e7b6 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:10.559354 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 833/1667] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:10.563005 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a574f-bca6-4ba3-a620-142e0524c9b4 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:10.563571 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a574f-bca6-4ba3-a620-142e0524c9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:10.564080 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a574f-bca6-4ba3-a620-142e0524c9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:10.602758 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-ec9a574f-bca6-4ba3-a620-142e0524c9b4 None None] MFA Rules not processed for user `d606fd7ef269460a8110223932f02d19`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:10.605899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ec9a574f-bca6-4ba3-a620-142e0524c9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:10.608492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 835/1668] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:46:09 2022] DELETE /identity/v3/users/c14e84a3885d439d941d40c67defb0b8 => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:10.609876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d40c1-853e-4fb0-b923-be7a4a358a15 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:10.612178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-517d40c1-853e-4fb0-b923-be7a4a358a15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:10.767903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d40c1-853e-4fb0-b923-be7a4a358a15 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:10.783770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a700c9-9ae0-41fa-9227-d1ccb597e417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:10.784512 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 834/1669] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Jul 18 08:46:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:10.784718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a700c9-9ae0-41fa-9227-d1ccb597e417 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:10.785010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a700c9-9ae0-41fa-9227-d1ccb597e417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:10.785289 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a700c9-9ae0-41fa-9227-d1ccb597e417 admin admin] PATH_INFO: `/v3/projects/b2320c44ac1c4954ab40fb192e9a4254/users/0349a014e40f422bb1b5a6150f60cf60/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:10.787472 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a700c9-9ae0-41fa-9227-d1ccb597e417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2320c44ac1c4954ab40fb192e9a4254, user_id=0349a014e40f422bb1b5a6150f60cf60, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:10.790281 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a700c9-9ae0-41fa-9227-d1ccb597e417 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:10.840768 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 836/1670] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:46:10 2022] PUT /identity/v3/projects/b2320c44ac1c4954ab40fb192e9a4254/users/0349a014e40f422bb1b5a6150f60cf60/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:10.848854 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f1802-55be-42af-853e-fb9d5940d4bd None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:10.852349 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e763c55-1b5b-40bf-9427-ceefeb6a917e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:10.853106 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f1802-55be-42af-853e-fb9d5940d4bd None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:10.853489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e763c55-1b5b-40bf-9427-ceefeb6a917e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:10.853706 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f1802-55be-42af-853e-fb9d5940d4bd None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:10.853915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e763c55-1b5b-40bf-9427-ceefeb6a917e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:10.855514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 835/1671] 10.222.0.57 () {54 vars in 961 bytes} [Mon Jul 18 08:46:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:46:10.869685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e32f7-7cc6-4903-a319-54564456df42 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:10.870216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e32f7-7cc6-4903-a319-54564456df42 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:10.870646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e32f7-7cc6-4903-a319-54564456df42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:10.933891 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-355e32f7-7cc6-4903-a319-54564456df42 None None] MFA Rules not processed for user `d606fd7ef269460a8110223932f02d19`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:10.936692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0e763c55-1b5b-40bf-9427-ceefeb6a917e None None] MFA Rules not processed for user `0349a014e40f422bb1b5a6150f60cf60`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:10.938041 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0e763c55-1b5b-40bf-9427-ceefeb6a917e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:10.939332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-355e32f7-7cc6-4903-a319-54564456df42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:11.150169 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 837/1672] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:10 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:11.164739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e763c55-1b5b-40bf-9427-ceefeb6a917e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:11.166839 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0e763c55-1b5b-40bf-9427-ceefeb6a917e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:11.308558 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 836/1673] 10.222.0.57 () {58 vars in 1058 bytes} [Mon Jul 18 08:46:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:46:11.310773 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e32f7-7cc6-4903-a319-54564456df42 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:11.414093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e32f7-7cc6-4903-a319-54564456df42 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:11.425289 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b93748-a69f-4b75-8424-247be7900336 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd606fd7ef269460a8110223932f02d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:11.426149 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b93748-a69f-4b75-8424-247be7900336 service placement] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:11.426600 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b93748-a69f-4b75-8424-247be7900336 service placement] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:11.426992 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b93748-a69f-4b75-8424-247be7900336 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:11.428788 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e763c55-1b5b-40bf-9427-ceefeb6a917e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:11.449375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b93748-a69f-4b75-8424-247be7900336 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:11.449749 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e600043-f34e-416a-b707-00e3c1de19e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:11.450986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e600043-f34e-416a-b707-00e3c1de19e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:11.451525 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e600043-f34e-416a-b707-00e3c1de19e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:11.452394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e600043-f34e-416a-b707-00e3c1de19e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:11.453062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b93748-a69f-4b75-8424-247be7900336 service placement] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:11.485801 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 837/1674] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:46:11.491623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e600043-f34e-416a-b707-00e3c1de19e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:11.498583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e600043-f34e-416a-b707-00e3c1de19e1 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:11.554622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 838/1675] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:11 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:46:11.648394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:11.949163 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b93748-a69f-4b75-8424-247be7900336 service placement] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:12.056474 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b93748-a69f-4b75-8424-247be7900336 service placement] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:12.097134 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e5b15-2ec0-4c80-b6fd-f6f2e66be50d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:12.097134 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e5b15-2ec0-4c80-b6fd-f6f2e66be50d service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:12.097134 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e5b15-2ec0-4c80-b6fd-f6f2e66be50d service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:12.097134 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e5b15-2ec0-4c80-b6fd-f6f2e66be50d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:12.097461 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e81e5b15-2ec0-4c80-b6fd-f6f2e66be50d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:12.177246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:12.314587 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81e5b15-2ec0-4c80-b6fd-f6f2e66be50d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:12.316488 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81e5b15-2ec0-4c80-b6fd-f6f2e66be50d service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:12.355469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 838/1676] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:11 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:46:13.530239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e600043-f34e-416a-b707-00e3c1de19e1 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:13.532682 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4e600043-f34e-416a-b707-00e3c1de19e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:13.665244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e5b15-2ec0-4c80-b6fd-f6f2e66be50d service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:13.669117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e81e5b15-2ec0-4c80-b6fd-f6f2e66be50d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:13.942922 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e600043-f34e-416a-b707-00e3c1de19e1 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:13.943017 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e5b15-2ec0-4c80-b6fd-f6f2e66be50d service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:13.950055 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86a49e-baca-4411-944f-3ae1e5165893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:13.950263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71198fc6-8954-4686-a8b9-162db1939def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:13.951762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71198fc6-8954-4686-a8b9-162db1939def admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:13.952141 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86a49e-baca-4411-944f-3ae1e5165893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:13.952552 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86a49e-baca-4411-944f-3ae1e5165893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:13.952937 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86a49e-baca-4411-944f-3ae1e5165893 admin admin] PATH_INFO: `/v3/projects/f507254c7200433abbc276fb95fcbb61` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:13.954868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86a49e-baca-4411-944f-3ae1e5165893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f507254c7200433abbc276fb95fcbb61)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:13.957147 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86a49e-baca-4411-944f-3ae1e5165893 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:13.960157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71198fc6-8954-4686-a8b9-162db1939def admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:13.960458 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71198fc6-8954-4686-a8b9-162db1939def admin admin] PATH_INFO: `/v3/users/1ee0885d9e78456a90a30b5aa245ea64` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:13.965355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71198fc6-8954-4686-a8b9-162db1939def admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ee0885d9e78456a90a30b5aa245ea64)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:13.968644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71198fc6-8954-4686-a8b9-162db1939def admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:14.304889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ab86a49e-baca-4411-944f-3ae1e5165893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f507254c7200433abbc276fb95fcbb61', 'request_id': 'req-ab86a49e-baca-4411-944f-3ae1e5165893'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:14.342978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ab86a49e-baca-4411-944f-3ae1e5165893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f507254c7200433abbc276fb95fcbb61', 'request_id': 'req-ab86a49e-baca-4411-944f-3ae1e5165893'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:14.344639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-71198fc6-8954-4686-a8b9-162db1939def admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ee0885d9e78456a90a30b5aa245ea64'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:14.345518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-71198fc6-8954-4686-a8b9-162db1939def admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ee0885d9e78456a90a30b5aa245ea64'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:14.347197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-71198fc6-8954-4686-a8b9-162db1939def admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ee0885d9e78456a90a30b5aa245ea64'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:14.377302 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ab86a49e-baca-4411-944f-3ae1e5165893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f507254c7200433abbc276fb95fcbb61', 'request_id': 'req-ab86a49e-baca-4411-944f-3ae1e5165893'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:14.387768 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-71198fc6-8954-4686-a8b9-162db1939def admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ee0885d9e78456a90a30b5aa245ea64'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:14.425980 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-71198fc6-8954-4686-a8b9-162db1939def admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ee0885d9e78456a90a30b5aa245ea64'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:14.469905 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ab86a49e-baca-4411-944f-3ae1e5165893 admin admin] The token cache is being invalidate because project f507254c7200433abbc276fb95fcbb61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:46:14.470211 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ab86a49e-baca-4411-944f-3ae1e5165893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:14.480664 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ab86a49e-baca-4411-944f-3ae1e5165893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:14.488541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 839/1677] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:46:13 2022] DELETE /identity/v3/projects/f507254c7200433abbc276fb95fcbb61 => generated 0 bytes in 824 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:14.490220 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01719e24-3ce6-4569-b117-dc87b559468c None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:14.490522 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01719e24-3ce6-4569-b117-dc87b559468c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:14.492249 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01719e24-3ce6-4569-b117-dc87b559468c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:14.571234 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-01719e24-3ce6-4569-b117-dc87b559468c None None] MFA Rules not processed for user `67a5cc6144c0434b9f16fb050c4c7662`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:14.574972 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-01719e24-3ce6-4569-b117-dc87b559468c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:14.614718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 839/1678] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:46:13 2022] DELETE /identity/v3/users/1ee0885d9e78456a90a30b5aa245ea64 => generated 0 bytes in 1084 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:46:14.624368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a48f5-6a41-40d8-aeb1-4be30b7ada09 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:14.624368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a48f5-6a41-40d8-aeb1-4be30b7ada09 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:14.624368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a48f5-6a41-40d8-aeb1-4be30b7ada09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:14.680731 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b19a48f5-6a41-40d8-aeb1-4be30b7ada09 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:14.682019 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b19a48f5-6a41-40d8-aeb1-4be30b7ada09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:14.710681 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:14.855113 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:14.917583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 840/1679] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:14.935538 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670cde4-ab61-4829-b533-fda827e53e02 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:14.938934 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670cde4-ab61-4829-b533-fda827e53e02 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:14.939440 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670cde4-ab61-4829-b533-fda827e53e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:14.973207 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 840/1680] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:14.984611 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861d447-1995-423f-9c45-ba41562fdaf7 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:14.985342 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861d447-1995-423f-9c45-ba41562fdaf7 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:14.985930 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861d447-1995-423f-9c45-ba41562fdaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:15.018048 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3670cde4-ab61-4829-b533-fda827e53e02 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:15.025560 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3670cde4-ab61-4829-b533-fda827e53e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:15.044920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0861d447-1995-423f-9c45-ba41562fdaf7 None None] MFA Rules not processed for user `67a5cc6144c0434b9f16fb050c4c7662`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:15.049482 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0861d447-1995-423f-9c45-ba41562fdaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:15.085393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 841/1681] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:15.090185 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670cde4-ab61-4829-b533-fda827e53e02 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:15.155245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670cde4-ab61-4829-b533-fda827e53e02 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:15.182352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ce76d-4dde-40ec-9d8b-6428dfa0fa2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:15.188118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ce76d-4dde-40ec-9d8b-6428dfa0fa2b admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:15.196161 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ce76d-4dde-40ec-9d8b-6428dfa0fa2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:15.196161 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ce76d-4dde-40ec-9d8b-6428dfa0fa2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:15.196161 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ce76d-4dde-40ec-9d8b-6428dfa0fa2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:46:15.196161 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ce76d-4dde-40ec-9d8b-6428dfa0fa2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:15.211220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ce76d-4dde-40ec-9d8b-6428dfa0fa2b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:15.277574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 841/1682] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:15.281731 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 842/1683] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:46:15.286454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861d447-1995-423f-9c45-ba41562fdaf7 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:15.324379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861d447-1995-423f-9c45-ba41562fdaf7 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:15.344064 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe143c-95bb-4328-b1dc-3cd68e2ed8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:15.344776 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe143c-95bb-4328-b1dc-3cd68e2ed8bf admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:15.345065 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe143c-95bb-4328-b1dc-3cd68e2ed8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:15.345348 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe143c-95bb-4328-b1dc-3cd68e2ed8bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:15.346124 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe143c-95bb-4328-b1dc-3cd68e2ed8bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:15.352566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb373646-3e2f-473d-b4a2-579a57711d0b None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:15.355892 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb373646-3e2f-473d-b4a2-579a57711d0b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:15.355892 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb373646-3e2f-473d-b4a2-579a57711d0b None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:15.356250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 843/1684] 10.222.0.57 () {54 vars in 962 bytes} [Mon Jul 18 08:46:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:46:15.361058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe143c-95bb-4328-b1dc-3cd68e2ed8bf admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:15.382479 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0a858-4d3a-48ae-937e-ef80a7035f26 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:15.383108 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0a858-4d3a-48ae-937e-ef80a7035f26 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:15.383874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0a858-4d3a-48ae-937e-ef80a7035f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:15.568290 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 842/1685] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:46:15 2022] POST /identity/v3/projects => generated 383 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:46:15.568861 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-edc0a858-4d3a-48ae-937e-ef80a7035f26 None None] MFA Rules not processed for user `84cba474fc7c470ea119de697b9d7ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:15.571292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-edc0a858-4d3a-48ae-937e-ef80a7035f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:15.573695 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe143c-95bb-4328-b1dc-3cd68e2ed8bf admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:15.874527 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe143c-95bb-4328-b1dc-3cd68e2ed8bf admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:15.890041 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 844/1686] 10.222.0.57 () {58 vars in 1059 bytes} [Mon Jul 18 08:46:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 509 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:15.890041 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0a858-4d3a-48ae-937e-ef80a7035f26 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:15.898416 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af77f5-90ec-4028-9770-6a70bdb71719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:15.899112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af77f5-90ec-4028-9770-6a70bdb71719 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:15.899394 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af77f5-90ec-4028-9770-6a70bdb71719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:15.900971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-edc0a858-4d3a-48ae-937e-ef80a7035f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:15.902188 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af77f5-90ec-4028-9770-6a70bdb71719 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:15.903196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af77f5-90ec-4028-9770-6a70bdb71719 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:15.909765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af77f5-90ec-4028-9770-6a70bdb71719 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:15.962417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0a858-4d3a-48ae-937e-ef80a7035f26 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:15.990103 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e3546-fc1d-4bed-927a-d8e666498d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:15.991028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e3546-fc1d-4bed-927a-d8e666498d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:15.994131 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e3546-fc1d-4bed-927a-d8e666498d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:15.994640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e3546-fc1d-4bed-927a-d8e666498d8d admin admin] PATH_INFO: `/v3/projects/324cdcf46dde4aa987a2907fb89712fa` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:15.999188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e3546-fc1d-4bed-927a-d8e666498d8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=324cdcf46dde4aa987a2907fb89712fa)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:16.001753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e3546-fc1d-4bed-927a-d8e666498d8d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:16.066141 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 843/1687] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:46:15 2022] POST /identity/v3/users => generated 292 bytes in 488 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:46:16.066141 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af77f5-90ec-4028-9770-6a70bdb71719 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:16.124904 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af77f5-90ec-4028-9770-6a70bdb71719 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:16.144497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22840142-9d5c-42c3-a0f9-dc35c49ac9fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:16.148431 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22840142-9d5c-42c3-a0f9-dc35c49ac9fd service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:16.158826 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22840142-9d5c-42c3-a0f9-dc35c49ac9fd service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:16.158826 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22840142-9d5c-42c3-a0f9-dc35c49ac9fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:16.298275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:16.347458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22840142-9d5c-42c3-a0f9-dc35c49ac9fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:16.349412 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22840142-9d5c-42c3-a0f9-dc35c49ac9fd service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:16.381298 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 844/1688] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:46:16.382419 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22840142-9d5c-42c3-a0f9-dc35c49ac9fd service nova] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:16.414293 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22840142-9d5c-42c3-a0f9-dc35c49ac9fd service nova] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:16.442142 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-692e3546-fc1d-4bed-927a-d8e666498d8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '324cdcf46dde4aa987a2907fb89712fa', 'request_id': 'req-692e3546-fc1d-4bed-927a-d8e666498d8d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:16.485834 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-692e3546-fc1d-4bed-927a-d8e666498d8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '324cdcf46dde4aa987a2907fb89712fa', 'request_id': 'req-692e3546-fc1d-4bed-927a-d8e666498d8d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:16.509918 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-692e3546-fc1d-4bed-927a-d8e666498d8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '324cdcf46dde4aa987a2907fb89712fa', 'request_id': 'req-692e3546-fc1d-4bed-927a-d8e666498d8d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:16.638197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-692e3546-fc1d-4bed-927a-d8e666498d8d admin admin] The token cache is being invalidate because project 324cdcf46dde4aa987a2907fb89712fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:46:16.642572 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-692e3546-fc1d-4bed-927a-d8e666498d8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:16.646073 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-692e3546-fc1d-4bed-927a-d8e666498d8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:16.665282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 845/1689] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:46:15 2022] DELETE /identity/v3/projects/324cdcf46dde4aa987a2907fb89712fa => generated 0 bytes in 775 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:16.686154 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186a92f-9a59-4666-8c6d-4f6b583382df None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:16.686154 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186a92f-9a59-4666-8c6d-4f6b583382df None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:16.686154 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186a92f-9a59-4666-8c6d-4f6b583382df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:16.732422 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357641ab-2cdf-4cb8-931f-1f76b9439f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:16.736238 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357641ab-2cdf-4cb8-931f-1f76b9439f1e admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:16.741725 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357641ab-2cdf-4cb8-931f-1f76b9439f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:16.743778 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357641ab-2cdf-4cb8-931f-1f76b9439f1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:16.744772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357641ab-2cdf-4cb8-931f-1f76b9439f1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:16.752520 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357641ab-2cdf-4cb8-931f-1f76b9439f1e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:16.793854 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 845/1690] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:16.797455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357641ab-2cdf-4cb8-931f-1f76b9439f1e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:16.810630 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-357641ab-2cdf-4cb8-931f-1f76b9439f1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:16.855726 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357641ab-2cdf-4cb8-931f-1f76b9439f1e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:16.878779 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089600c-ee52-4f0f-8b8d-7557f7b2c441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:16.880581 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089600c-ee52-4f0f-8b8d-7557f7b2c441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:16.881356 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089600c-ee52-4f0f-8b8d-7557f7b2c441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:16.890548 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089600c-ee52-4f0f-8b8d-7557f7b2c441 admin admin] PATH_INFO: `/v3/users/b43d349d945649a396292ec7033af9dd` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:16.894329 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089600c-ee52-4f0f-8b8d-7557f7b2c441 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b43d349d945649a396292ec7033af9dd)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:16.898074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089600c-ee52-4f0f-8b8d-7557f7b2c441 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:16.909584 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a186a92f-9a59-4666-8c6d-4f6b583382df None None] MFA Rules not processed for user `88b4bd52ede644f0aac8dff94e09dd7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:16.929857 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a186a92f-9a59-4666-8c6d-4f6b583382df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:17.022430 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:17.175870 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 846/1691] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 510 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:17.203913 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186a92f-9a59-4666-8c6d-4f6b583382df None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:17.208392 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d089600c-ee52-4f0f-8b8d-7557f7b2c441 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b43d349d945649a396292ec7033af9dd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:17.209120 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d089600c-ee52-4f0f-8b8d-7557f7b2c441 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b43d349d945649a396292ec7033af9dd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:17.212510 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a186a92f-9a59-4666-8c6d-4f6b583382df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:17.217036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d089600c-ee52-4f0f-8b8d-7557f7b2c441 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b43d349d945649a396292ec7033af9dd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:17.265010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d089600c-ee52-4f0f-8b8d-7557f7b2c441 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b43d349d945649a396292ec7033af9dd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:17.269899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186a92f-9a59-4666-8c6d-4f6b583382df None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:17.289990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6c942-3b35-407b-9bec-24b4bbedd5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:17.292748 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6c942-3b35-407b-9bec-24b4bbedd5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:17.295848 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6c942-3b35-407b-9bec-24b4bbedd5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:17.297670 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d089600c-ee52-4f0f-8b8d-7557f7b2c441 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b43d349d945649a396292ec7033af9dd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:17.297819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6c942-3b35-407b-9bec-24b4bbedd5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:17.299017 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad6c942-3b35-407b-9bec-24b4bbedd5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:17.301799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad6c942-3b35-407b-9bec-24b4bbedd5e6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:17.317159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 847/1692] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:17.319157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69daee0-bbbe-437b-8ddb-651f5b2f26f6 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:17.319621 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69daee0-bbbe-437b-8ddb-651f5b2f26f6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:17.320107 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69daee0-bbbe-437b-8ddb-651f5b2f26f6 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:17.328929 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 848/1693] 10.222.0.57 () {54 vars in 962 bytes} [Mon Jul 18 08:46:17 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:46:17.330113 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69daee0-bbbe-437b-8ddb-651f5b2f26f6 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:17.334342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c69daee0-bbbe-437b-8ddb-651f5b2f26f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:17.353070 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 846/1694] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:46:16 2022] DELETE /identity/v3/users/b43d349d945649a396292ec7033af9dd => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:17.354886 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b56f9-8934-434f-aa47-ab5623fa679a None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:17.358638 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b56f9-8934-434f-aa47-ab5623fa679a None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:17.360199 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69daee0-bbbe-437b-8ddb-651f5b2f26f6 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:17.360700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b56f9-8934-434f-aa47-ab5623fa679a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:17.431492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-445b56f9-8934-434f-aa47-ab5623fa679a None None] MFA Rules not processed for user `84cba474fc7c470ea119de697b9d7ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:17.439542 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-445b56f9-8934-434f-aa47-ab5623fa679a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:17.553482 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310400b8-a997-437d-8d66-0917cc0efa50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:17.555384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310400b8-a997-437d-8d66-0917cc0efa50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:17.555874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310400b8-a997-437d-8d66-0917cc0efa50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:17.556648 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310400b8-a997-437d-8d66-0917cc0efa50 admin admin] PATH_INFO: `/v3/projects/27c7c13f3f3f473a8c95f8f730e3f2bd/users/3f4c6a3284414332b80f870710d77e87/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:17.598741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:17.604071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310400b8-a997-437d-8d66-0917cc0efa50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27c7c13f3f3f473a8c95f8f730e3f2bd, user_id=3f4c6a3284414332b80f870710d77e87, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:17.612625 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310400b8-a997-437d-8d66-0917cc0efa50 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:17.720511 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 847/1695] 10.222.0.57 () {58 vars in 1059 bytes} [Mon Jul 18 08:46:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:17.728552 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b17ed9-30b7-40c4-9127-9e579e9a72e0 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:17.730770 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b17ed9-30b7-40c4-9127-9e579e9a72e0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:17.731154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b17ed9-30b7-40c4-9127-9e579e9a72e0 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:17.740269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 848/1696] 10.222.0.57 () {54 vars in 912 bytes} [Mon Jul 18 08:46:17 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:46:17.745587 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b17ed9-30b7-40c4-9127-9e579e9a72e0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:17.750793 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 849/1697] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:46:17 2022] PUT /identity/v3/projects/27c7c13f3f3f473a8c95f8f730e3f2bd/users/3f4c6a3284414332b80f870710d77e87/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:17.752604 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5712db-8abf-4c6e-8fb5-9ca697c9dff6 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:17.752899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5712db-8abf-4c6e-8fb5-9ca697c9dff6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:17.756233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5712db-8abf-4c6e-8fb5-9ca697c9dff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:17.819568 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0a5712db-8abf-4c6e-8fb5-9ca697c9dff6 None None] MFA Rules not processed for user `84cba474fc7c470ea119de697b9d7ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:17.824012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0a5712db-8abf-4c6e-8fb5-9ca697c9dff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:18.049179 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b17ed9-30b7-40c4-9127-9e579e9a72e0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:18.064484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 850/1698] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Jul 18 08:46:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:18.066189 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d241d-1d5d-4988-8f8b-5547c846631e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:18.068882 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865dc16-c4eb-4507-ae1a-31b9ee0e2586 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:18.069536 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d241d-1d5d-4988-8f8b-5547c846631e service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:18.069935 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d241d-1d5d-4988-8f8b-5547c846631e service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:18.070653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d241d-1d5d-4988-8f8b-5547c846631e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:18.070852 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865dc16-c4eb-4507-ae1a-31b9ee0e2586 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:18.074226 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865dc16-c4eb-4507-ae1a-31b9ee0e2586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:18.077220 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d49d241d-1d5d-4988-8f8b-5547c846631e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:18.177931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d865dc16-c4eb-4507-ae1a-31b9ee0e2586 None None] MFA Rules not processed for user `3f4c6a3284414332b80f870710d77e87`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:18.182105 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d865dc16-c4eb-4507-ae1a-31b9ee0e2586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:18.270724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:18.302316 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d241d-1d5d-4988-8f8b-5547c846631e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:18.305418 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d241d-1d5d-4988-8f8b-5547c846631e service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:18.332150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 849/1699] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:46:18.347484 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480b6a3-1f1e-4ce5-8e8a-af1a19e52c23 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:18.349337 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480b6a3-1f1e-4ce5-8e8a-af1a19e52c23 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:18.349672 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480b6a3-1f1e-4ce5-8e8a-af1a19e52c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:18.511288 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 851/1700] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:18.514179 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865dc16-c4eb-4507-ae1a-31b9ee0e2586 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:18.528732 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d865dc16-c4eb-4507-ae1a-31b9ee0e2586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:18.550932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0480b6a3-1f1e-4ce5-8e8a-af1a19e52c23 None None] MFA Rules not processed for user `4e6b788e0819483580c09902a341ea6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:18.556285 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0480b6a3-1f1e-4ce5-8e8a-af1a19e52c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:18.917459 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 850/1701] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 573 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:18.919857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480b6a3-1f1e-4ce5-8e8a-af1a19e52c23 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:18.937298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865dc16-c4eb-4507-ae1a-31b9ee0e2586 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:18.955061 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e26794-f8a8-4490-88d9-03c4355b2105 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '694088ed30b94bad9d3b16f3766252df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:18.958147 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e26794-f8a8-4490-88d9-03c4355b2105 service glance] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:18.962661 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e26794-f8a8-4490-88d9-03c4355b2105 service glance] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:18.962994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e26794-f8a8-4490-88d9-03c4355b2105 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:18.992912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:19.040603 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e26794-f8a8-4490-88d9-03c4355b2105 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:19.046845 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e26794-f8a8-4490-88d9-03c4355b2105 service glance] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:19.100476 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 852/1702] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:46:19.105845 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e26794-f8a8-4490-88d9-03c4355b2105 service glance] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:19.111975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-72e26794-f8a8-4490-88d9-03c4355b2105 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:19.197264 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480b6a3-1f1e-4ce5-8e8a-af1a19e52c23 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:19.232484 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e0f18-e4e8-4816-8011-952447a43555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:19.237217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e0f18-e4e8-4816-8011-952447a43555 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:19.237217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e0f18-e4e8-4816-8011-952447a43555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:19.237217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e0f18-e4e8-4816-8011-952447a43555 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:19.237217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9e0f18-e4e8-4816-8011-952447a43555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:19.242090 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9e0f18-e4e8-4816-8011-952447a43555 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:19.328239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e26794-f8a8-4490-88d9-03c4355b2105 service glance] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:19.334536 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc3954-6dda-42bd-8c44-87d0327ede54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:19.335520 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc3954-6dda-42bd-8c44-87d0327ede54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:19.335840 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc3954-6dda-42bd-8c44-87d0327ede54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:19.336118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc3954-6dda-42bd-8c44-87d0327ede54 admin admin] PATH_INFO: `/v3/projects/bb330e7e37f2447a9ec98db9dd4c5a9e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:19.337589 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc3954-6dda-42bd-8c44-87d0327ede54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb330e7e37f2447a9ec98db9dd4c5a9e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:19.339779 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc3954-6dda-42bd-8c44-87d0327ede54 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:19.361795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 851/1703] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:46:18 2022] POST /identity/v3/projects => generated 383 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:46:19.366712 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e0f18-e4e8-4816-8011-952447a43555 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:19.582171 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e0f18-e4e8-4816-8011-952447a43555 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:19.612838 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:19.624027 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-50cc3954-6dda-42bd-8c44-87d0327ede54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb330e7e37f2447a9ec98db9dd4c5a9e', 'request_id': 'req-50cc3954-6dda-42bd-8c44-87d0327ede54'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:19.654938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-50cc3954-6dda-42bd-8c44-87d0327ede54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb330e7e37f2447a9ec98db9dd4c5a9e', 'request_id': 'req-50cc3954-6dda-42bd-8c44-87d0327ede54'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:19.687553 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-50cc3954-6dda-42bd-8c44-87d0327ede54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb330e7e37f2447a9ec98db9dd4c5a9e', 'request_id': 'req-50cc3954-6dda-42bd-8c44-87d0327ede54'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:19.772599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9250ff2-eb5c-4822-8e3d-cd96ab7bc717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:19.773762 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9250ff2-eb5c-4822-8e3d-cd96ab7bc717 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:19.774179 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9250ff2-eb5c-4822-8e3d-cd96ab7bc717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:19.774540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9250ff2-eb5c-4822-8e3d-cd96ab7bc717 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:19.776055 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9250ff2-eb5c-4822-8e3d-cd96ab7bc717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:19.778904 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9250ff2-eb5c-4822-8e3d-cd96ab7bc717 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:19.873767 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-50cc3954-6dda-42bd-8c44-87d0327ede54 admin admin] The token cache is being invalidate because project bb330e7e37f2447a9ec98db9dd4c5a9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:46:19.874418 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-50cc3954-6dda-42bd-8c44-87d0327ede54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:19.880277 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-50cc3954-6dda-42bd-8c44-87d0327ede54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:19.882926 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 853/1704] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:46:19 2022] DELETE /identity/v3/projects/bb330e7e37f2447a9ec98db9dd4c5a9e => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:19.890517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8eaf90-e05b-468b-af53-57d4e7f427ee None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:19.896396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8eaf90-e05b-468b-af53-57d4e7f427ee None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:19.903978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 852/1705] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:46:19 2022] POST /identity/v3/users => generated 298 bytes in 538 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:46:19.907579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e25c97-1ba3-4324-b5f2-32674b59bcf0 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:19.907579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e25c97-1ba3-4324-b5f2-32674b59bcf0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:19.907579 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e25c97-1ba3-4324-b5f2-32674b59bcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:19.908441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8eaf90-e05b-468b-af53-57d4e7f427ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:19.973726 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-72e25c97-1ba3-4324-b5f2-32674b59bcf0 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:19.975009 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-72e25c97-1ba3-4324-b5f2-32674b59bcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:20.007047 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1d8eaf90-e05b-468b-af53-57d4e7f427ee None None] MFA Rules not processed for user `94eed85a730b46dcb1cf694a3ebf2d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:20.012610 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1d8eaf90-e05b-468b-af53-57d4e7f427ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:20.032189 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 853/1706] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:20.032189 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e25c97-1ba3-4324-b5f2-32674b59bcf0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:20.041590 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-72e25c97-1ba3-4324-b5f2-32674b59bcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:20.159983 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:20.199072 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 854/1707] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:20.200966 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8eaf90-e05b-468b-af53-57d4e7f427ee None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:20.208907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1d8eaf90-e05b-468b-af53-57d4e7f427ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:20.238007 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8eaf90-e05b-468b-af53-57d4e7f427ee None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:20.240520 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e25c97-1ba3-4324-b5f2-32674b59bcf0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:20.244684 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a54a38-343b-4f3b-af15-cdf33e088017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:20.245584 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a54a38-343b-4f3b-af15-cdf33e088017 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:20.246539 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a54a38-343b-4f3b-af15-cdf33e088017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:20.247846 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a54a38-343b-4f3b-af15-cdf33e088017 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:20.249447 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a54a38-343b-4f3b-af15-cdf33e088017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:20.255828 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a54a38-343b-4f3b-af15-cdf33e088017 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:20.256710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a6b99-a053-45db-8e21-ff9239c3fa0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:20.259294 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a6b99-a053-45db-8e21-ff9239c3fa0d service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:20.259774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a6b99-a053-45db-8e21-ff9239c3fa0d service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:20.262731 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a6b99-a053-45db-8e21-ff9239c3fa0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:20.266928 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f85a6b99-a053-45db-8e21-ff9239c3fa0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:20.320176 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 855/1708] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:20.323090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f4837-b4c0-4978-aefd-43f9f5ec4c21 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:20.324118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f4837-b4c0-4978-aefd-43f9f5ec4c21 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:20.324844 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f4837-b4c0-4978-aefd-43f9f5ec4c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:20.388942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-653f4837-b4c0-4978-aefd-43f9f5ec4c21 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:20.393388 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-653f4837-b4c0-4978-aefd-43f9f5ec4c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:20.411242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 856/1709] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:20.421802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c99bcd-289e-4893-94a5-2f58d30d6810 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:20.421802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c99bcd-289e-4893-94a5-2f58d30d6810 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:20.421802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c99bcd-289e-4893-94a5-2f58d30d6810 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:20.443705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 857/1710] 10.222.0.57 () {54 vars in 912 bytes} [Mon Jul 18 08:46:20 2022] GET /identity => generated 270 bytes in 31 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:46:20.445703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c99bcd-289e-4893-94a5-2f58d30d6810 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:20.472644 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a6b99-a053-45db-8e21-ff9239c3fa0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:20.474495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a6b99-a053-45db-8e21-ff9239c3fa0d service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:20.489128 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 854/1711] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:46:20.492424 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a6b99-a053-45db-8e21-ff9239c3fa0d service nova] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:20.504400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a6b99-a053-45db-8e21-ff9239c3fa0d service nova] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:20.519726 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c5683-9aea-4711-a524-b0463d9c8906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:20.520691 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c99bcd-289e-4893-94a5-2f58d30d6810 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:20.526825 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c5683-9aea-4711-a524-b0463d9c8906 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:20.526825 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c5683-9aea-4711-a524-b0463d9c8906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:20.526825 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c5683-9aea-4711-a524-b0463d9c8906 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:20.540313 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c5683-9aea-4711-a524-b0463d9c8906 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:46:20.540640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c5683-9aea-4711-a524-b0463d9c8906 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:20.542362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c5683-9aea-4711-a524-b0463d9c8906 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:20.555574 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d2d0b-26e1-4eea-bbfb-4fa51ed10268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:20.555907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d2d0b-26e1-4eea-bbfb-4fa51ed10268 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:20.556272 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d2d0b-26e1-4eea-bbfb-4fa51ed10268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:20.556466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d2d0b-26e1-4eea-bbfb-4fa51ed10268 admin admin] PATH_INFO: `/v3/projects/64e7c75d844d4e21838529464b006cab/users/59c005b78d2c4e2d9e2464351db5c32c/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:20.599839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 855/1712] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:46:20.601476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b47f0-f7a2-4c07-a880-87390ec3b9f5 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:20.601771 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b47f0-f7a2-4c07-a880-87390ec3b9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:20.602055 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b47f0-f7a2-4c07-a880-87390ec3b9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:20.616322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d2d0b-26e1-4eea-bbfb-4fa51ed10268 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e7c75d844d4e21838529464b006cab, user_id=59c005b78d2c4e2d9e2464351db5c32c, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:20.619764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d2d0b-26e1-4eea-bbfb-4fa51ed10268 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:20.696929 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0d3b47f0-f7a2-4c07-a880-87390ec3b9f5 None None] MFA Rules not processed for user `24a6b794f5774e35b67e9ec4358ffc66`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:20.698544 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0d3b47f0-f7a2-4c07-a880-87390ec3b9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:20.741823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 858/1713] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:46:20 2022] PUT /identity/v3/projects/64e7c75d844d4e21838529464b006cab/users/59c005b78d2c4e2d9e2464351db5c32c/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:20.742650 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d2d0b-26e1-4eea-bbfb-4fa51ed10268 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:20.868836 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 856/1714] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Jul 18 08:46:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:20.871217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b47f0-f7a2-4c07-a880-87390ec3b9f5 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:20.907530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b47f0-f7a2-4c07-a880-87390ec3b9f5 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:20.922901 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d2d0b-26e1-4eea-bbfb-4fa51ed10268 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:20.939460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882ee54-438e-40c4-b85d-177d764ff555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:20.940439 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882ee54-438e-40c4-b85d-177d764ff555 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:20.940583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882ee54-438e-40c4-b85d-177d764ff555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:20.940890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882ee54-438e-40c4-b85d-177d764ff555 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:20.941682 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882ee54-438e-40c4-b85d-177d764ff555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:20.948318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882ee54-438e-40c4-b85d-177d764ff555 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:21.002964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:21.066272 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edaff9-d52c-4ac3-9ca0-c3854b85b9a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:21.069748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edaff9-d52c-4ac3-9ca0-c3854b85b9a3 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:21.069748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edaff9-d52c-4ac3-9ca0-c3854b85b9a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:21.070135 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edaff9-d52c-4ac3-9ca0-c3854b85b9a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:21.111911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 859/1715] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:46:20 2022] POST /identity/v3/projects => generated 373 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:46:21.113081 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882ee54-438e-40c4-b85d-177d764ff555 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:21.138926 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:21.229768 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882ee54-438e-40c4-b85d-177d764ff555 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:21.242718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d28036-22e1-468a-9bc3-fe91969225c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:21.243785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d28036-22e1-468a-9bc3-fe91969225c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:21.244190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d28036-22e1-468a-9bc3-fe91969225c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:21.244613 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d28036-22e1-468a-9bc3-fe91969225c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:21.245574 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d28036-22e1-468a-9bc3-fe91969225c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:21.248122 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d28036-22e1-468a-9bc3-fe91969225c2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:21.271844 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 860/1716] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:21.273509 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d28036-22e1-468a-9bc3-fe91969225c2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:21.280829 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c6d28036-22e1-468a-9bc3-fe91969225c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:21.282810 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edaff9-d52c-4ac3-9ca0-c3854b85b9a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:21.289422 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edaff9-d52c-4ac3-9ca0-c3854b85b9a3 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:21.333833 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 857/1717] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:46:21.356587 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edaff9-d52c-4ac3-9ca0-c3854b85b9a3 service nova] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:21.386080 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edaff9-d52c-4ac3-9ca0-c3854b85b9a3 service nova] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:21.416392 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e7e2f-442e-4410-ab08-c2ce6512f0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:21.417116 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e7e2f-442e-4410-ab08-c2ce6512f0c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:21.417538 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e7e2f-442e-4410-ab08-c2ce6512f0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:21.417685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e7e2f-442e-4410-ab08-c2ce6512f0c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:21.418530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e7e2f-442e-4410-ab08-c2ce6512f0c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:21.443746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d28036-22e1-468a-9bc3-fe91969225c2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:21.443997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e7e2f-442e-4410-ab08-c2ce6512f0c0 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:21.454020 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72746e6-4a7a-4e0d-8549-d17cae0685df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:21.458348 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72746e6-4a7a-4e0d-8549-d17cae0685df service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:21.458348 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72746e6-4a7a-4e0d-8549-d17cae0685df service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:21.458713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72746e6-4a7a-4e0d-8549-d17cae0685df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:21.501890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72746e6-4a7a-4e0d-8549-d17cae0685df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:21.509920 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72746e6-4a7a-4e0d-8549-d17cae0685df service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:21.536674 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 861/1718] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:46:21.538768 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72746e6-4a7a-4e0d-8549-d17cae0685df service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:21.573431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72746e6-4a7a-4e0d-8549-d17cae0685df service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:21.583502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c91c52-0dad-48bf-b318-ea7bbbf4c3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:21.584839 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c91c52-0dad-48bf-b318-ea7bbbf4c3bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:21.585129 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c91c52-0dad-48bf-b318-ea7bbbf4c3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:21.585409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c91c52-0dad-48bf-b318-ea7bbbf4c3bc admin admin] PATH_INFO: `/v3/projects/64e7c75d844d4e21838529464b006cab/users/59c005b78d2c4e2d9e2464351db5c32c/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:21.596477 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c91c52-0dad-48bf-b318-ea7bbbf4c3bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e7c75d844d4e21838529464b006cab, user_id=59c005b78d2c4e2d9e2464351db5c32c, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:21.600335 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c91c52-0dad-48bf-b318-ea7bbbf4c3bc admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:21.621829 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 858/1719] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:46:21 2022] POST /identity/v3/users => generated 287 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:46:21.626794 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f2005-a369-434e-89ef-fe55a02a1a0c None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:21.628042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f2005-a369-434e-89ef-fe55a02a1a0c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:21.628534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f2005-a369-434e-89ef-fe55a02a1a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:21.644264 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 862/1720] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:46:21 2022] PUT /identity/v3/projects/64e7c75d844d4e21838529464b006cab/users/59c005b78d2c4e2d9e2464351db5c32c/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:21.644264 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c91c52-0dad-48bf-b318-ea7bbbf4c3bc admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:21.685531 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-bc4f2005-a369-434e-89ef-fe55a02a1a0c None None] MFA Rules not processed for user `56b3d8d145d94b358c86345eb5c3b306`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:21.687737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bc4f2005-a369-434e-89ef-fe55a02a1a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:21.804891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c91c52-0dad-48bf-b318-ea7bbbf4c3bc admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:21.824573 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59ae8e-7199-4fea-b842-5b5d55eb5dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:21.825461 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59ae8e-7199-4fea-b842-5b5d55eb5dba admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:21.825678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59ae8e-7199-4fea-b842-5b5d55eb5dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:21.826173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59ae8e-7199-4fea-b842-5b5d55eb5dba admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:21.827406 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59ae8e-7199-4fea-b842-5b5d55eb5dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:21.829128 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59ae8e-7199-4fea-b842-5b5d55eb5dba admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:21.847472 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 863/1721] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:21.849111 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca630b-848d-4d63-9c67-0fe4c01f5eaa None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:21.849403 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca630b-848d-4d63-9c67-0fe4c01f5eaa None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:21.849683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca630b-848d-4d63-9c67-0fe4c01f5eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:21.891789 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 859/1722] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:21.900704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f2005-a369-434e-89ef-fe55a02a1a0c None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:21.910149 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1cca630b-848d-4d63-9c67-0fe4c01f5eaa None None] MFA Rules not processed for user `59c005b78d2c4e2d9e2464351db5c32c`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:21.914615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1cca630b-848d-4d63-9c67-0fe4c01f5eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:21.937943 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:21.942266 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f2005-a369-434e-89ef-fe55a02a1a0c None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:21.972725 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf17b03-3b37-42ea-b9eb-2274526b1da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:21.973586 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf17b03-3b37-42ea-b9eb-2274526b1da6 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:21.973881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf17b03-3b37-42ea-b9eb-2274526b1da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:21.974165 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf17b03-3b37-42ea-b9eb-2274526b1da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:21.977507 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf17b03-3b37-42ea-b9eb-2274526b1da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:21.979250 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf17b03-3b37-42ea-b9eb-2274526b1da6 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:22.024006 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 860/1723] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:22.030408 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61271d51-7fde-4f3d-a49d-116fa8fcd08b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:22.030822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61271d51-7fde-4f3d-a49d-116fa8fcd08b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:22.031109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61271d51-7fde-4f3d-a49d-116fa8fcd08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:22.105704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-61271d51-7fde-4f3d-a49d-116fa8fcd08b None None] MFA Rules not processed for user `56b3d8d145d94b358c86345eb5c3b306`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:22.116862 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-61271d51-7fde-4f3d-a49d-116fa8fcd08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:22.117053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 864/1724] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:22.119197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca630b-848d-4d63-9c67-0fe4c01f5eaa None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:22.134904 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 861/1725] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:22.142554 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61271d51-7fde-4f3d-a49d-116fa8fcd08b None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:22.146277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-61271d51-7fde-4f3d-a49d-116fa8fcd08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:22.149644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca630b-848d-4d63-9c67-0fe4c01f5eaa None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:22.167143 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcaa02-edc2-4574-af72-2e561108a2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:22.169851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcaa02-edc2-4574-af72-2e561108a2fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:22.170133 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcaa02-edc2-4574-af72-2e561108a2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:22.173543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcaa02-edc2-4574-af72-2e561108a2fc admin admin] PATH_INFO: `/v3/projects/bf93e90ca0364aebb603729b54a3b84e/users/da50b3d5815e4e9e9f6c4a3d36da071c/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:22.252837 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:22.253365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fcaa02-edc2-4574-af72-2e561108a2fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf93e90ca0364aebb603729b54a3b84e, user_id=da50b3d5815e4e9e9f6c4a3d36da071c, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:22.256912 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fcaa02-edc2-4574-af72-2e561108a2fc admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:22.279466 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61271d51-7fde-4f3d-a49d-116fa8fcd08b None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:22.285300 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da91a9a-76dc-406c-afba-21f33f83c15b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:22.286320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da91a9a-76dc-406c-afba-21f33f83c15b service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:22.286651 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da91a9a-76dc-406c-afba-21f33f83c15b service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:22.286946 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da91a9a-76dc-406c-afba-21f33f83c15b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:22.300372 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da91a9a-76dc-406c-afba-21f33f83c15b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:22.302592 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da91a9a-76dc-406c-afba-21f33f83c15b service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:22.316557 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 862/1726] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:46:22.317536 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da91a9a-76dc-406c-afba-21f33f83c15b service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:22.393299 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 865/1727] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:46:22 2022] PUT /identity/v3/projects/bf93e90ca0364aebb603729b54a3b84e/users/da50b3d5815e4e9e9f6c4a3d36da071c/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:22.409921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6072dd-171e-4be3-8245-c4ba67c90133 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:22.410220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6072dd-171e-4be3-8245-c4ba67c90133 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:22.410911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6072dd-171e-4be3-8245-c4ba67c90133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:22.442345 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da91a9a-76dc-406c-afba-21f33f83c15b service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:22.539524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-9e6072dd-171e-4be3-8245-c4ba67c90133 None None] MFA Rules not processed for user `da50b3d5815e4e9e9f6c4a3d36da071c`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:22.543403 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9e6072dd-171e-4be3-8245-c4ba67c90133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:22.622143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e3564-97d9-411d-9102-81a58c0bab02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:22.623051 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e3564-97d9-411d-9102-81a58c0bab02 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:22.623407 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e3564-97d9-411d-9102-81a58c0bab02 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:22.623882 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e3564-97d9-411d-9102-81a58c0bab02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:22.776658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e3564-97d9-411d-9102-81a58c0bab02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:22.781481 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e3564-97d9-411d-9102-81a58c0bab02 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:22.872216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 863/1728] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:46:22.878253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 866/1729] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:46:22.881165 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e3564-97d9-411d-9102-81a58c0bab02 service nova] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:22.917582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6072dd-171e-4be3-8245-c4ba67c90133 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:23.000708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:23.219703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6072dd-171e-4be3-8245-c4ba67c90133 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:23.235537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace72a0-0420-4156-a57c-aa392a8f15e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:23.236423 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace72a0-0420-4156-a57c-aa392a8f15e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:23.236717 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace72a0-0420-4156-a57c-aa392a8f15e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:23.236998 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace72a0-0420-4156-a57c-aa392a8f15e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:23.247834 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3ace72a0-0420-4156-a57c-aa392a8f15e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:23.250675 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e3564-97d9-411d-9102-81a58c0bab02 service nova] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:23.269497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e2c8a-7f70-4e08-a197-9797e5116124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:23.270214 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e2c8a-7f70-4e08-a197-9797e5116124 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:23.270504 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e2c8a-7f70-4e08-a197-9797e5116124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:23.270781 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e2c8a-7f70-4e08-a197-9797e5116124 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:23.271524 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e2c8a-7f70-4e08-a197-9797e5116124 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:23.291815 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e2c8a-7f70-4e08-a197-9797e5116124 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:23.341336 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:23.399005 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace72a0-0420-4156-a57c-aa392a8f15e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:23.400705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace72a0-0420-4156-a57c-aa392a8f15e5 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:23.427351 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 867/1730] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:46:23.429655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccc5c2-08b0-4e47-8d84-328b2252c277 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:23.430208 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccc5c2-08b0-4e47-8d84-328b2252c277 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:23.430687 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccc5c2-08b0-4e47-8d84-328b2252c277 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:23.432961 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 868/1731] 10.222.0.57 () {54 vars in 912 bytes} [Mon Jul 18 08:46:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:46:23.440215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4b143-e3bc-4741-ac3c-1f5441b5ae57 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:23.440634 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4b143-e3bc-4741-ac3c-1f5441b5ae57 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:23.441025 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4b143-e3bc-4741-ac3c-1f5441b5ae57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:23.458338 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 864/1732] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:46:22 2022] POST /identity/v3/projects => generated 371 bytes in 578 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:46:23.467925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e2c8a-7f70-4e08-a197-9797e5116124 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:23.525685 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c0f4b143-e3bc-4741-ac3c-1f5441b5ae57 None None] MFA Rules not processed for user `84cba474fc7c470ea119de697b9d7ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:23.534457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c0f4b143-e3bc-4741-ac3c-1f5441b5ae57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:23.875345 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e2c8a-7f70-4e08-a197-9797e5116124 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:23.910721 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2dd93-d54b-48ba-a48b-7579c0d85779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:23.914621 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2dd93-d54b-48ba-a48b-7579c0d85779 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:23.920166 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2dd93-d54b-48ba-a48b-7579c0d85779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:23.925430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2dd93-d54b-48ba-a48b-7579c0d85779 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:23.925430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2dd93-d54b-48ba-a48b-7579c0d85779 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:23.934463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2dd93-d54b-48ba-a48b-7579c0d85779 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:24.087208 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 869/1733] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Jul 18 08:46:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 649 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:24.098489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4b143-e3bc-4741-ac3c-1f5441b5ae57 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:24.104008 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c0f4b143-e3bc-4741-ac3c-1f5441b5ae57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:24.152161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 865/1734] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:46:23 2022] POST /identity/v3/users => generated 292 bytes in 688 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:46:24.157355 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2dd93-d54b-48ba-a48b-7579c0d85779 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:24.165543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-dcd2dd93-d54b-48ba-a48b-7579c0d85779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:24.353540 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4b143-e3bc-4741-ac3c-1f5441b5ae57 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:24.367168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eec8b1-ea58-4d6d-9c7b-4b8628b187d0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '694088ed30b94bad9d3b16f3766252df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:24.369506 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eec8b1-ea58-4d6d-9c7b-4b8628b187d0 service glance] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:24.370809 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eec8b1-ea58-4d6d-9c7b-4b8628b187d0 service glance] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:24.373594 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eec8b1-ea58-4d6d-9c7b-4b8628b187d0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:24.398221 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eec8b1-ea58-4d6d-9c7b-4b8628b187d0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:24.401669 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eec8b1-ea58-4d6d-9c7b-4b8628b187d0 service glance] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:24.405360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2dd93-d54b-48ba-a48b-7579c0d85779 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:24.412489 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b56ffe-d2c2-4910-8466-4fde1f7ca9a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd606fd7ef269460a8110223932f02d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:24.413194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b56ffe-d2c2-4910-8466-4fde1f7ca9a8 service placement] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:24.413808 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b56ffe-d2c2-4910-8466-4fde1f7ca9a8 service placement] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:24.414097 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b56ffe-d2c2-4910-8466-4fde1f7ca9a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:24.418340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c2b56ffe-d2c2-4910-8466-4fde1f7ca9a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:24.434828 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 870/1735] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:46:24.441406 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eec8b1-ea58-4d6d-9c7b-4b8628b187d0 service glance] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:24.476475 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b56ffe-d2c2-4910-8466-4fde1f7ca9a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:24.479960 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b56ffe-d2c2-4910-8466-4fde1f7ca9a8 service placement] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:24.492437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eec8b1-ea58-4d6d-9c7b-4b8628b187d0 service glance] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:24.513637 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883f183-e067-42c4-80ff-864813407614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:24.514864 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883f183-e067-42c4-80ff-864813407614 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:24.515414 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883f183-e067-42c4-80ff-864813407614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:24.515914 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883f183-e067-42c4-80ff-864813407614 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:24.517971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883f183-e067-42c4-80ff-864813407614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:24.530381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883f183-e067-42c4-80ff-864813407614 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:24.534255 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 866/1736] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:46:24.595872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 871/1737] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:24.601016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b56ffe-d2c2-4910-8466-4fde1f7ca9a8 service placement] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:24.654369 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b56ffe-d2c2-4910-8466-4fde1f7ca9a8 service placement] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:24.688182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851f9a6-05c2-4da1-a4fe-da45df8e00a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:24.689795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851f9a6-05c2-4da1-a4fe-da45df8e00a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:24.690225 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851f9a6-05c2-4da1-a4fe-da45df8e00a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:24.696535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851f9a6-05c2-4da1-a4fe-da45df8e00a9 admin admin] PATH_INFO: `/v3/projects/c747b26e490d4910ad598f7b58efe530/users/d2b09cb6e12c47338d73df57982a6376/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:24.824833 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:24.850347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851f9a6-05c2-4da1-a4fe-da45df8e00a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c747b26e490d4910ad598f7b58efe530, user_id=d2b09cb6e12c47338d73df57982a6376, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:24.870244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851f9a6-05c2-4da1-a4fe-da45df8e00a9 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:25.088797 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 867/1738] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:46:24 2022] PUT /identity/v3/projects/c747b26e490d4910ad598f7b58efe530/users/d2b09cb6e12c47338d73df57982a6376/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:46:25.095449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883f183-e067-42c4-80ff-864813407614 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:25.108396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883f183-e067-42c4-80ff-864813407614 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:25.411173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:25.439638 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e328f92-566a-42d4-8e19-644f0ba8264b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:25.442955 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e328f92-566a-42d4-8e19-644f0ba8264b admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:25.449795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e328f92-566a-42d4-8e19-644f0ba8264b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:25.452635 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e328f92-566a-42d4-8e19-644f0ba8264b admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:25.455118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e328f92-566a-42d4-8e19-644f0ba8264b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:25.461299 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e328f92-566a-42d4-8e19-644f0ba8264b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:25.492374 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 872/1739] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:25.507452 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851f9a6-05c2-4da1-a4fe-da45df8e00a9 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:25.543902 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851f9a6-05c2-4da1-a4fe-da45df8e00a9 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:25.553487 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998d829-60c3-4695-8733-5ac4de8c6e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:25.555366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998d829-60c3-4695-8733-5ac4de8c6e84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:25.556016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998d829-60c3-4695-8733-5ac4de8c6e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:25.560949 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998d829-60c3-4695-8733-5ac4de8c6e84 admin admin] PATH_INFO: `/v3/projects/c747b26e490d4910ad598f7b58efe530/users/d2b09cb6e12c47338d73df57982a6376/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:25.562429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998d829-60c3-4695-8733-5ac4de8c6e84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c747b26e490d4910ad598f7b58efe530, user_id=d2b09cb6e12c47338d73df57982a6376, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:25.572513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998d829-60c3-4695-8733-5ac4de8c6e84 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:25.659084 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 868/1740] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:46:25 2022] PUT /identity/v3/projects/c747b26e490d4910ad598f7b58efe530/users/d2b09cb6e12c47338d73df57982a6376/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:25.666627 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3b80f-e0ae-4a31-911f-be117631cf72 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:25.667190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3b80f-e0ae-4a31-911f-be117631cf72 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:25.667550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3b80f-e0ae-4a31-911f-be117631cf72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:25.743087 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5cc3b80f-e0ae-4a31-911f-be117631cf72 None None] MFA Rules not processed for user `d2b09cb6e12c47338d73df57982a6376`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:25.743976 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5cc3b80f-e0ae-4a31-911f-be117631cf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:25.988563 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 873/1741] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:25 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:26.016589 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998d829-60c3-4695-8733-5ac4de8c6e84 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:26.210299 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998d829-60c3-4695-8733-5ac4de8c6e84 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:26.225520 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9146ef1-0786-4ff2-89d7-478977a047cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:26.227209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9146ef1-0786-4ff2-89d7-478977a047cc service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:26.227941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9146ef1-0786-4ff2-89d7-478977a047cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:26.228875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9146ef1-0786-4ff2-89d7-478977a047cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:26.256803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9146ef1-0786-4ff2-89d7-478977a047cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:26.260922 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9146ef1-0786-4ff2-89d7-478977a047cc service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:26.302149 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 869/1742] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:26 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:46:26.662280 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3b80f-e0ae-4a31-911f-be117631cf72 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:26.664184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5cc3b80f-e0ae-4a31-911f-be117631cf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:26.692978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3b80f-e0ae-4a31-911f-be117631cf72 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:26.699350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5934ca-18df-466b-829e-44f7c8523ed4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:26.700367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5934ca-18df-466b-829e-44f7c8523ed4 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:26.700627 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5934ca-18df-466b-829e-44f7c8523ed4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:26.701035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5934ca-18df-466b-829e-44f7c8523ed4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:26.909965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5934ca-18df-466b-829e-44f7c8523ed4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:26.910855 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5934ca-18df-466b-829e-44f7c8523ed4 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:26.937430 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 874/1743] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:46:27.191175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad9fd0-6ba3-4c07-ae57-1822b8943af4 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:27.192176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad9fd0-6ba3-4c07-ae57-1822b8943af4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:27.192634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad9fd0-6ba3-4c07-ae57-1822b8943af4 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:27.194748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 870/1744] 10.222.0.57 () {54 vars in 912 bytes} [Mon Jul 18 08:46:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:46:27.209715 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335191ca-d62f-4871-a38d-b7745986ab2d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:27.210007 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335191ca-d62f-4871-a38d-b7745986ab2d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:27.210290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335191ca-d62f-4871-a38d-b7745986ab2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:27.262569 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-335191ca-d62f-4871-a38d-b7745986ab2d None None] MFA Rules not processed for user `24a6b794f5774e35b67e9ec4358ffc66`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:27.267720 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-335191ca-d62f-4871-a38d-b7745986ab2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:27.456052 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 875/1745] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Jul 18 08:46:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:27.485841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad9fd0-6ba3-4c07-ae57-1822b8943af4 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:27.489142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a4ad9fd0-6ba3-4c07-ae57-1822b8943af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:27.518607 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad9fd0-6ba3-4c07-ae57-1822b8943af4 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:27.527166 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08987ce9-60ae-4e42-964a-851f5fb9bab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:27.528564 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08987ce9-60ae-4e42-964a-851f5fb9bab4 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:27.529812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08987ce9-60ae-4e42-964a-851f5fb9bab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:27.530276 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08987ce9-60ae-4e42-964a-851f5fb9bab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:27.551483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08987ce9-60ae-4e42-964a-851f5fb9bab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:27.556015 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08987ce9-60ae-4e42-964a-851f5fb9bab4 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:27.578645 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 871/1746] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:46:28.390023 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e1b3f-0c32-4838-8dd8-db2249cc5e4a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:28.390555 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e1b3f-0c32-4838-8dd8-db2249cc5e4a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:28.390764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e1b3f-0c32-4838-8dd8-db2249cc5e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:28.430035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c71e1b3f-0c32-4838-8dd8-db2249cc5e4a None None] MFA Rules not processed for user `88b4bd52ede644f0aac8dff94e09dd7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:28.431437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c71e1b3f-0c32-4838-8dd8-db2249cc5e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:28.488744 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:28.577564 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 876/1747] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:28.968750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08987ce9-60ae-4e42-964a-851f5fb9bab4 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:29.237143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08987ce9-60ae-4e42-964a-851f5fb9bab4 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:29.259070 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c6ac9-ae45-4af1-b032-d012da9cb950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:29.261570 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c6ac9-ae45-4af1-b032-d012da9cb950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:29.262005 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c6ac9-ae45-4af1-b032-d012da9cb950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:29.262274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c6ac9-ae45-4af1-b032-d012da9cb950 admin admin] PATH_INFO: `/v3/users/3f4c6a3284414332b80f870710d77e87` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:29.266051 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f8dab-3d90-4189-b47f-326743390b57 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:29.267036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c6ac9-ae45-4af1-b032-d012da9cb950 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f4c6a3284414332b80f870710d77e87)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:29.269219 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c6ac9-ae45-4af1-b032-d012da9cb950 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:29.270145 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f8dab-3d90-4189-b47f-326743390b57 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:29.270716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f8dab-3d90-4189-b47f-326743390b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:29.371486 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-df2f8dab-3d90-4189-b47f-326743390b57 None None] MFA Rules not processed for user `56b3d8d145d94b358c86345eb5c3b306`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:29.382890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-df2f8dab-3d90-4189-b47f-326743390b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:29.639156 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-081c6ac9-ae45-4af1-b032-d012da9cb950 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f4c6a3284414332b80f870710d77e87'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:29.640883 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-081c6ac9-ae45-4af1-b032-d012da9cb950 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f4c6a3284414332b80f870710d77e87'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:29.643250 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-081c6ac9-ae45-4af1-b032-d012da9cb950 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f4c6a3284414332b80f870710d77e87'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:29.674092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 877/1748] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:29.675708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6b766-c09a-40bc-ac8a-084c31a94793 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:29.677725 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6b766-c09a-40bc-ac8a-084c31a94793 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:29.679366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-081c6ac9-ae45-4af1-b032-d012da9cb950 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f4c6a3284414332b80f870710d77e87'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:29.680548 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6b766-c09a-40bc-ac8a-084c31a94793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:29.693018 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:29.698525 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-081c6ac9-ae45-4af1-b032-d012da9cb950 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f4c6a3284414332b80f870710d77e87'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:29.719148 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a5b6b766-c09a-40bc-ac8a-084c31a94793 None None] MFA Rules not processed for user `94eed85a730b46dcb1cf694a3ebf2d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:29.723429 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a5b6b766-c09a-40bc-ac8a-084c31a94793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:29.853041 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 872/1749] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:46:28 2022] DELETE /identity/v3/users/3f4c6a3284414332b80f870710d77e87 => generated 0 bytes in 882 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:29.853041 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334321ff-200b-4976-a9cd-e49f8925b4b7 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:29.853041 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334321ff-200b-4976-a9cd-e49f8925b4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:29.853421 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334321ff-200b-4976-a9cd-e49f8925b4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:29.921720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-334321ff-200b-4976-a9cd-e49f8925b4b7 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:29.923036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-334321ff-200b-4976-a9cd-e49f8925b4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:29.957639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 878/1750] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:30.183577 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 873/1751] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:30.206641 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6b766-c09a-40bc-ac8a-084c31a94793 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:30.211213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a5b6b766-c09a-40bc-ac8a-084c31a94793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:30.271381 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6b766-c09a-40bc-ac8a-084c31a94793 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:30.532411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa51151-f579-436d-ba5a-d7ca65f69ca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:30.533870 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51151-f579-436d-ba5a-d7ca65f69ca9 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:30.534329 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51151-f579-436d-ba5a-d7ca65f69ca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:30.534813 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51151-f579-436d-ba5a-d7ca65f69ca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:30.573259 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa51151-f579-436d-ba5a-d7ca65f69ca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:30.579990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa51151-f579-436d-ba5a-d7ca65f69ca9 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:30.691058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 879/1752] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:46:31.195406 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e8b20-d3a7-40f3-b6dd-2142403ab77e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:31.196358 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e8b20-d3a7-40f3-b6dd-2142403ab77e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:31.196848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e8b20-d3a7-40f3-b6dd-2142403ab77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:31.249595 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-368e8b20-d3a7-40f3-b6dd-2142403ab77e None None] MFA Rules not processed for user `4e6b788e0819483580c09902a341ea6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:31.257119 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-368e8b20-d3a7-40f3-b6dd-2142403ab77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:31.267197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd53b4-afeb-40af-b78a-302357fcb9cd None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:31.267605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd53b4-afeb-40af-b78a-302357fcb9cd None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:31.268037 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd53b4-afeb-40af-b78a-302357fcb9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:31.448867 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:31.486111 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5dbd53b4-afeb-40af-b78a-302357fcb9cd None None] MFA Rules not processed for user `2b03f8b8c51e472bbd4cf0afc9c6b632`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:31.489433 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5dbd53b4-afeb-40af-b78a-302357fcb9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:31.528988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 874/1753] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:31.596933 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 880/1754] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Jul 18 08:46:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:32.230384 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e8b20-d3a7-40f3-b6dd-2142403ab77e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:32.230384 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-368e8b20-d3a7-40f3-b6dd-2142403ab77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:32.406649 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e8b20-d3a7-40f3-b6dd-2142403ab77e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:32.421166 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cac9a-1a1d-4b87-926d-0314983d7118 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09a5c2e804504038817cad76544dce5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:32.425315 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cac9a-1a1d-4b87-926d-0314983d7118 service swift] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:32.426209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cac9a-1a1d-4b87-926d-0314983d7118 service swift] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:32.426673 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cac9a-1a1d-4b87-926d-0314983d7118 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:32.454339 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cac9a-1a1d-4b87-926d-0314983d7118 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:32.456371 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cac9a-1a1d-4b87-926d-0314983d7118 service swift] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:32.485916 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 875/1755] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Jul 18 08:46:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 08:46:33.056906 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd53b4-afeb-40af-b78a-302357fcb9cd None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:33.068642 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5dbd53b4-afeb-40af-b78a-302357fcb9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:33.161396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd53b4-afeb-40af-b78a-302357fcb9cd None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:33.215177 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94336e79-d1a3-4e45-bdde-675d16deb2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:33.220292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94336e79-d1a3-4e45-bdde-675d16deb2f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:33.224629 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94336e79-d1a3-4e45-bdde-675d16deb2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:33.227116 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94336e79-d1a3-4e45-bdde-675d16deb2f4 admin admin] PATH_INFO: `/v3/projects/27c7c13f3f3f473a8c95f8f730e3f2bd` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:33.244589 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94336e79-d1a3-4e45-bdde-675d16deb2f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27c7c13f3f3f473a8c95f8f730e3f2bd)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:33.289435 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94336e79-d1a3-4e45-bdde-675d16deb2f4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:33.647125 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-94336e79-d1a3-4e45-bdde-675d16deb2f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27c7c13f3f3f473a8c95f8f730e3f2bd', 'request_id': 'req-94336e79-d1a3-4e45-bdde-675d16deb2f4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:33.687917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-94336e79-d1a3-4e45-bdde-675d16deb2f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27c7c13f3f3f473a8c95f8f730e3f2bd', 'request_id': 'req-94336e79-d1a3-4e45-bdde-675d16deb2f4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:33.735459 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-94336e79-d1a3-4e45-bdde-675d16deb2f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27c7c13f3f3f473a8c95f8f730e3f2bd', 'request_id': 'req-94336e79-d1a3-4e45-bdde-675d16deb2f4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:33.881420 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-94336e79-d1a3-4e45-bdde-675d16deb2f4 admin admin] The token cache is being invalidate because project 27c7c13f3f3f473a8c95f8f730e3f2bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:46:33.881745 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-94336e79-d1a3-4e45-bdde-675d16deb2f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:33.885171 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-94336e79-d1a3-4e45-bdde-675d16deb2f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:33.898928 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 881/1756] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:46:33 2022] DELETE /identity/v3/projects/27c7c13f3f3f473a8c95f8f730e3f2bd => generated 0 bytes in 838 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:33.901382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cac9a-1a1d-4b87-926d-0314983d7118 service swift] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:33.910543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:33.926359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cac9a-1a1d-4b87-926d-0314983d7118 service swift] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:34.368893 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4bc11-9662-44cc-ad1d-16d11915b503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:34.373793 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4bc11-9662-44cc-ad1d-16d11915b503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:34.374393 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4bc11-9662-44cc-ad1d-16d11915b503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:34.374828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4bc11-9662-44cc-ad1d-16d11915b503 admin admin] PATH_INFO: `/v3/users/59c005b78d2c4e2d9e2464351db5c32c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:34.377885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4bc11-9662-44cc-ad1d-16d11915b503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59c005b78d2c4e2d9e2464351db5c32c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:34.382724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4bc11-9662-44cc-ad1d-16d11915b503 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:34.829900 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-63d4bc11-9662-44cc-ad1d-16d11915b503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59c005b78d2c4e2d9e2464351db5c32c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:34.834931 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-63d4bc11-9662-44cc-ad1d-16d11915b503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59c005b78d2c4e2d9e2464351db5c32c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:34.848409 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-63d4bc11-9662-44cc-ad1d-16d11915b503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59c005b78d2c4e2d9e2464351db5c32c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:34.940320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-63d4bc11-9662-44cc-ad1d-16d11915b503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59c005b78d2c4e2d9e2464351db5c32c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:34.972832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-63d4bc11-9662-44cc-ad1d-16d11915b503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59c005b78d2c4e2d9e2464351db5c32c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:35.106436 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 876/1757] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:46:33 2022] DELETE /identity/v3/users/59c005b78d2c4e2d9e2464351db5c32c => generated 0 bytes in 1207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:36.431876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94336e79-d1a3-4e45-bdde-675d16deb2f4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:36.434432 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-94336e79-d1a3-4e45-bdde-675d16deb2f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:36.496467 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94336e79-d1a3-4e45-bdde-675d16deb2f4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:36.660639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c7b34-aa0b-4fb5-b245-961ecb9f8854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:36.661645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c7b34-aa0b-4fb5-b245-961ecb9f8854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:36.662006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c7b34-aa0b-4fb5-b245-961ecb9f8854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:36.662363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c7b34-aa0b-4fb5-b245-961ecb9f8854 admin admin] PATH_INFO: `/v3/projects/64e7c75d844d4e21838529464b006cab` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:36.664367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c7b34-aa0b-4fb5-b245-961ecb9f8854 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64e7c75d844d4e21838529464b006cab)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:36.668140 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c7b34-aa0b-4fb5-b245-961ecb9f8854 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:36.719693 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:36.914114 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e79c7b34-aa0b-4fb5-b245-961ecb9f8854 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64e7c75d844d4e21838529464b006cab', 'request_id': 'req-e79c7b34-aa0b-4fb5-b245-961ecb9f8854'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:36.929126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e79c7b34-aa0b-4fb5-b245-961ecb9f8854 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64e7c75d844d4e21838529464b006cab', 'request_id': 'req-e79c7b34-aa0b-4fb5-b245-961ecb9f8854'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:36.947279 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e79c7b34-aa0b-4fb5-b245-961ecb9f8854 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64e7c75d844d4e21838529464b006cab', 'request_id': 'req-e79c7b34-aa0b-4fb5-b245-961ecb9f8854'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:37.576737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e79c7b34-aa0b-4fb5-b245-961ecb9f8854 admin admin] The token cache is being invalidate because project 64e7c75d844d4e21838529464b006cab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:46:37.576737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e79c7b34-aa0b-4fb5-b245-961ecb9f8854 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:37.579163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e79c7b34-aa0b-4fb5-b245-961ecb9f8854 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:37.582159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 882/1758] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:46:36 2022] DELETE /identity/v3/projects/64e7c75d844d4e21838529464b006cab => generated 0 bytes in 1150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:37.588395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07becbc5-796c-45e1-b802-fdf6dd0996c0 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:37.601287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07becbc5-796c-45e1-b802-fdf6dd0996c0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:37.605557 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07becbc5-796c-45e1-b802-fdf6dd0996c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:37.694839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-07becbc5-796c-45e1-b802-fdf6dd0996c0 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:37.696483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-07becbc5-796c-45e1-b802-fdf6dd0996c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:37.852484 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 877/1759] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:37.857837 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ce19b-ec1c-4728-a546-4bf349088ea4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:37.857837 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ce19b-ec1c-4728-a546-4bf349088ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:37.857837 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ce19b-ec1c-4728-a546-4bf349088ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:37.905458 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-503ce19b-ec1c-4728-a546-4bf349088ea4 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:37.909158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-503ce19b-ec1c-4728-a546-4bf349088ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:37.933785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 883/1760] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:37.942106 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07becbc5-796c-45e1-b802-fdf6dd0996c0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:37.984099 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07becbc5-796c-45e1-b802-fdf6dd0996c0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:38.013103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816413e2-bc17-4142-9edd-a5ff8f84b5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:38.017669 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816413e2-bc17-4142-9edd-a5ff8f84b5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:38.018963 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816413e2-bc17-4142-9edd-a5ff8f84b5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:38.019960 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816413e2-bc17-4142-9edd-a5ff8f84b5b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:38.023749 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816413e2-bc17-4142-9edd-a5ff8f84b5b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:46:38.025099 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816413e2-bc17-4142-9edd-a5ff8f84b5b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:38.030245 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816413e2-bc17-4142-9edd-a5ff8f84b5b4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:38.147744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 878/1761] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:46:38.150175 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ce19b-ec1c-4728-a546-4bf349088ea4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:38.177340 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ce19b-ec1c-4728-a546-4bf349088ea4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:38.195431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb202719-1284-4df8-94b3-82518a780f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:38.198549 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb202719-1284-4df8-94b3-82518a780f26 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:38.203238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb202719-1284-4df8-94b3-82518a780f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:38.203238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb202719-1284-4df8-94b3-82518a780f26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:38.204493 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb202719-1284-4df8-94b3-82518a780f26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:38.209189 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb202719-1284-4df8-94b3-82518a780f26 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:38.366466 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d0d8a-a6ce-4865-8ed1-8661ad14451d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:38.367229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d0d8a-a6ce-4865-8ed1-8661ad14451d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:38.367965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d0d8a-a6ce-4865-8ed1-8661ad14451d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:38.380693 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 884/1762] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:46:38 2022] POST /identity/v3/projects => generated 387 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:46:38.383957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb202719-1284-4df8-94b3-82518a780f26 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:38.471504 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6a4d0d8a-a6ce-4865-8ed1-8661ad14451d None None] MFA Rules not processed for user `24a6b794f5774e35b67e9ec4358ffc66`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:38.477127 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6a4d0d8a-a6ce-4865-8ed1-8661ad14451d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:38.516540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:38.565056 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb202719-1284-4df8-94b3-82518a780f26 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:38.598343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a055b5e-8770-4d15-bda5-da3cf90ca8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:38.599490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a055b5e-8770-4d15-bda5-da3cf90ca8c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:38.599490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a055b5e-8770-4d15-bda5-da3cf90ca8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:38.599988 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a055b5e-8770-4d15-bda5-da3cf90ca8c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:38.602330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a055b5e-8770-4d15-bda5-da3cf90ca8c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:38.606222 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a055b5e-8770-4d15-bda5-da3cf90ca8c1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:38.755597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 885/1763] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:46:38 2022] POST /identity/v3/users => generated 294 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:46:38.755863 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 879/1764] 10.222.0.47 () {58 vars in 1015 bytes} [Mon Jul 18 08:46:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:46:38.761080 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a055b5e-8770-4d15-bda5-da3cf90ca8c1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:38.803386 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d0d8a-a6ce-4865-8ed1-8661ad14451d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:38.803386 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6a4d0d8a-a6ce-4865-8ed1-8661ad14451d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:38.804365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a055b5e-8770-4d15-bda5-da3cf90ca8c1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:38.838132 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cf607-9f87-44d0-9429-6488c42aca55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:38.839281 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cf607-9f87-44d0-9429-6488c42aca55 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:38.839899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cf607-9f87-44d0-9429-6488c42aca55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:38.840445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cf607-9f87-44d0-9429-6488c42aca55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:38.842069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cf607-9f87-44d0-9429-6488c42aca55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:38.844252 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cf607-9f87-44d0-9429-6488c42aca55 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:38.860749 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d0d8a-a6ce-4865-8ed1-8661ad14451d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:38.896047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb641dbd-caca-49f2-b4c1-00de616c8e85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:38.897884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb641dbd-caca-49f2-b4c1-00de616c8e85 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:38.904292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 886/1765] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:38.906384 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb641dbd-caca-49f2-b4c1-00de616c8e85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:38.907028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cf607-9f87-44d0-9429-6488c42aca55 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:38.913008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb641dbd-caca-49f2-b4c1-00de616c8e85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:38.942520 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cf607-9f87-44d0-9429-6488c42aca55 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:38.974221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb641dbd-caca-49f2-b4c1-00de616c8e85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:38.975395 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cda137-77d4-4326-ba21-cc7c4db85773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:38.976291 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cda137-77d4-4326-ba21-cc7c4db85773 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:38.976696 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cda137-77d4-4326-ba21-cc7c4db85773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:38.977208 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cda137-77d4-4326-ba21-cc7c4db85773 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:38.978260 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cda137-77d4-4326-ba21-cc7c4db85773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:38.984027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb641dbd-caca-49f2-b4c1-00de616c8e85 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:38.992241 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cda137-77d4-4326-ba21-cc7c4db85773 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:39.033271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 880/1766] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:46:39.047500 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 887/1767] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:39.048920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb641dbd-caca-49f2-b4c1-00de616c8e85 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:39.147387 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb641dbd-caca-49f2-b4c1-00de616c8e85 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:39.194898 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249bf3f-1ab1-4467-9b60-db36a0f4c83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:39.208388 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249bf3f-1ab1-4467-9b60-db36a0f4c83f admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:39.209004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249bf3f-1ab1-4467-9b60-db36a0f4c83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:39.214145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249bf3f-1ab1-4467-9b60-db36a0f4c83f admin admin] PATH_INFO: `/v3/projects/e1a369037c7c429ebd67e30059035364/users/fe5a23542f5d45dbac6c45d995850cde/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:39.274551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249bf3f-1ab1-4467-9b60-db36a0f4c83f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1a369037c7c429ebd67e30059035364, user_id=fe5a23542f5d45dbac6c45d995850cde, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:39.279071 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249bf3f-1ab1-4467-9b60-db36a0f4c83f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:39.435836 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 881/1768] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:46:39 2022] PUT /identity/v3/projects/e1a369037c7c429ebd67e30059035364/users/fe5a23542f5d45dbac6c45d995850cde/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:39.446811 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363898ae-0204-421b-bfb5-602c3acb9057 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:39.447280 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363898ae-0204-421b-bfb5-602c3acb9057 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:39.447689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363898ae-0204-421b-bfb5-602c3acb9057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:39.531170 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-363898ae-0204-421b-bfb5-602c3acb9057 None None] MFA Rules not processed for user `fe5a23542f5d45dbac6c45d995850cde`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:39.534388 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-363898ae-0204-421b-bfb5-602c3acb9057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:39.676993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:39.759923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 888/1769] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:39.760540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249bf3f-1ab1-4467-9b60-db36a0f4c83f admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:39.947224 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249bf3f-1ab1-4467-9b60-db36a0f4c83f admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:39.958625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a6772-563c-403c-b96e-0e78c1adeed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:39.960814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a6772-563c-403c-b96e-0e78c1adeed6 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:39.961232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a6772-563c-403c-b96e-0e78c1adeed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:39.961787 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a6772-563c-403c-b96e-0e78c1adeed6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:39.963707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4a6772-563c-403c-b96e-0e78c1adeed6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:39.977334 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4a6772-563c-403c-b96e-0e78c1adeed6 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:40.144072 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 882/1770] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:46:39 2022] POST /identity/v3/projects => generated 389 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:46:40.145408 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363898ae-0204-421b-bfb5-602c3acb9057 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:40.162904 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363898ae-0204-421b-bfb5-602c3acb9057 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:40.399557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f73a2-315a-4530-be2d-47940c047c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:40.402486 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f73a2-315a-4530-be2d-47940c047c16 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:40.410098 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f73a2-315a-4530-be2d-47940c047c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:40.412554 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f73a2-315a-4530-be2d-47940c047c16 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:40.416342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f73a2-315a-4530-be2d-47940c047c16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:40.424897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f73a2-315a-4530-be2d-47940c047c16 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:40.591898 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a6772-563c-403c-b96e-0e78c1adeed6 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:40.592659 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 889/1771] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:46:40 2022] POST /identity/v3/users => generated 301 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:46:40.625398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a6772-563c-403c-b96e-0e78c1adeed6 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:40.651001 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589db0a8-402f-4184-9d88-50cce83b325d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:40.652180 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589db0a8-402f-4184-9d88-50cce83b325d admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:40.652766 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589db0a8-402f-4184-9d88-50cce83b325d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:40.653893 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589db0a8-402f-4184-9d88-50cce83b325d admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:40.660948 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589db0a8-402f-4184-9d88-50cce83b325d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:40.666094 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589db0a8-402f-4184-9d88-50cce83b325d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:40.710196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 883/1772] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:46:40.712521 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f73a2-315a-4530-be2d-47940c047c16 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:40.754148 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f73a2-315a-4530-be2d-47940c047c16 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:40.768785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e54dfa-8bff-4b2b-8753-cbe880221b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:40.770499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e54dfa-8bff-4b2b-8753-cbe880221b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:40.770499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e54dfa-8bff-4b2b-8753-cbe880221b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:40.770499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e54dfa-8bff-4b2b-8753-cbe880221b6b admin admin] PATH_INFO: `/v3/projects/1c5100b00c2144dc89f23d2ca98b4090/users/7263d7d21d5c4072a90126bf9262dedf/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:40.852616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e54dfa-8bff-4b2b-8753-cbe880221b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c5100b00c2144dc89f23d2ca98b4090, user_id=7263d7d21d5c4072a90126bf9262dedf, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:40.874633 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e54dfa-8bff-4b2b-8753-cbe880221b6b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:41.090259 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 890/1773] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:46:40 2022] PUT /identity/v3/projects/1c5100b00c2144dc89f23d2ca98b4090/users/7263d7d21d5c4072a90126bf9262dedf/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:41.093796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589db0a8-402f-4184-9d88-50cce83b325d admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:41.176971 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589db0a8-402f-4184-9d88-50cce83b325d admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:41.299170 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:41.309331 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b632b1-bfe1-476c-a0ca-4389f8759a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:41.311059 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b632b1-bfe1-476c-a0ca-4389f8759a49 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:41.311724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b632b1-bfe1-476c-a0ca-4389f8759a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:41.312156 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b632b1-bfe1-476c-a0ca-4389f8759a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:41.313072 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b632b1-bfe1-476c-a0ca-4389f8759a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:41.315042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b632b1-bfe1-476c-a0ca-4389f8759a49 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:41.329254 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 884/1774] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:41.334601 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e54dfa-8bff-4b2b-8753-cbe880221b6b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:41.407172 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e54dfa-8bff-4b2b-8753-cbe880221b6b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:41.447329 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784a9dc-d2e7-49d4-86d9-16b8c6b024bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:41.449265 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784a9dc-d2e7-49d4-86d9-16b8c6b024bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:41.449424 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784a9dc-d2e7-49d4-86d9-16b8c6b024bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:41.449866 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784a9dc-d2e7-49d4-86d9-16b8c6b024bb admin admin] PATH_INFO: `/v3/projects/1c5100b00c2144dc89f23d2ca98b4090/users/7263d7d21d5c4072a90126bf9262dedf/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:41.459382 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784a9dc-d2e7-49d4-86d9-16b8c6b024bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c5100b00c2144dc89f23d2ca98b4090, user_id=7263d7d21d5c4072a90126bf9262dedf, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:41.474943 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784a9dc-d2e7-49d4-86d9-16b8c6b024bb admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:41.568476 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 891/1775] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:46:41 2022] PUT /identity/v3/projects/1c5100b00c2144dc89f23d2ca98b4090/users/7263d7d21d5c4072a90126bf9262dedf/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:46:41.586992 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521fae0-7d1f-483e-ae20-be51998f18f3 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:41.587760 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521fae0-7d1f-483e-ae20-be51998f18f3 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:41.588294 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521fae0-7d1f-483e-ae20-be51998f18f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:41.713850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-8521fae0-7d1f-483e-ae20-be51998f18f3 None None] MFA Rules not processed for user `7263d7d21d5c4072a90126bf9262dedf`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:41.718052 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8521fae0-7d1f-483e-ae20-be51998f18f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:41.855961 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 885/1776] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:46:41.876792 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784a9dc-d2e7-49d4-86d9-16b8c6b024bb admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:41.881044 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c784a9dc-d2e7-49d4-86d9-16b8c6b024bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:42.216321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784a9dc-d2e7-49d4-86d9-16b8c6b024bb admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:42.245766 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f0fb1-9c5a-4946-a637-02ccb145f5b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:42.246534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f0fb1-9c5a-4946-a637-02ccb145f5b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:42.250289 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f0fb1-9c5a-4946-a637-02ccb145f5b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:42.250289 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f0fb1-9c5a-4946-a637-02ccb145f5b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:42.284700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f0fb1-9c5a-4946-a637-02ccb145f5b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:42.291390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f0fb1-9c5a-4946-a637-02ccb145f5b1 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:42.351203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 892/1777] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:46:42.679142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685495a9-8248-43c9-8502-07116d837510 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:42.683918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685495a9-8248-43c9-8502-07116d837510 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:42.683918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685495a9-8248-43c9-8502-07116d837510 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:42.683918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 886/1778] 10.222.0.57 () {54 vars in 921 bytes} [Mon Jul 18 08:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:46:42.955024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b9018-9db6-4e93-9c0b-fb5cd8586f7c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:42.955024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b9018-9db6-4e93-9c0b-fb5cd8586f7c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:42.955024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b9018-9db6-4e93-9c0b-fb5cd8586f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:43.024464 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-709b9018-9db6-4e93-9c0b-fb5cd8586f7c None None] MFA Rules not processed for user `84cba474fc7c470ea119de697b9d7ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:43.025761 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-709b9018-9db6-4e93-9c0b-fb5cd8586f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:43.242914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685495a9-8248-43c9-8502-07116d837510 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:43.248939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:43.420667 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 893/1779] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Jul 18 08:46:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:43.482934 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685495a9-8248-43c9-8502-07116d837510 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:43.495347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba124fe8-5bed-4f75-8add-9b85d11c1a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:43.497988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba124fe8-5bed-4f75-8add-9b85d11c1a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:43.498573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba124fe8-5bed-4f75-8add-9b85d11c1a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:43.499103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba124fe8-5bed-4f75-8add-9b85d11c1a2a admin admin] PATH_INFO: `/v3/users/fe5a23542f5d45dbac6c45d995850cde` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:43.501931 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba124fe8-5bed-4f75-8add-9b85d11c1a2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe5a23542f5d45dbac6c45d995850cde)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:43.507573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba124fe8-5bed-4f75-8add-9b85d11c1a2a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:44.308980 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b9018-9db6-4e93-9c0b-fb5cd8586f7c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:44.312161 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-709b9018-9db6-4e93-9c0b-fb5cd8586f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:44.398216 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b9018-9db6-4e93-9c0b-fb5cd8586f7c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:44.432678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2aff8-19b2-47b7-98d1-dafac4966e16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:44.434083 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2aff8-19b2-47b7-98d1-dafac4966e16 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:44.435430 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2aff8-19b2-47b7-98d1-dafac4966e16 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:44.440003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2aff8-19b2-47b7-98d1-dafac4966e16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:44.448511 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ba124fe8-5bed-4f75-8add-9b85d11c1a2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe5a23542f5d45dbac6c45d995850cde'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:44.456327 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ba124fe8-5bed-4f75-8add-9b85d11c1a2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe5a23542f5d45dbac6c45d995850cde'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:44.460033 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ba124fe8-5bed-4f75-8add-9b85d11c1a2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe5a23542f5d45dbac6c45d995850cde'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:44.472588 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2aff8-19b2-47b7-98d1-dafac4966e16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:44.475152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2aff8-19b2-47b7-98d1-dafac4966e16 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:44.498583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ba124fe8-5bed-4f75-8add-9b85d11c1a2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe5a23542f5d45dbac6c45d995850cde'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:44.516998 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 894/1780] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:46:44.523258 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ba124fe8-5bed-4f75-8add-9b85d11c1a2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe5a23542f5d45dbac6c45d995850cde'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:44.526294 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:44.679689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 887/1781] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:46:43 2022] DELETE /identity/v3/users/fe5a23542f5d45dbac6c45d995850cde => generated 0 bytes in 1436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:44.685685 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027ecbd-abec-4852-bce0-e6786f71c2db None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:44.689426 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027ecbd-abec-4852-bce0-e6786f71c2db None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:44.690244 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027ecbd-abec-4852-bce0-e6786f71c2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:44.737568 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e027ecbd-abec-4852-bce0-e6786f71c2db None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:44.750638 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e027ecbd-abec-4852-bce0-e6786f71c2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:45.093584 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 895/1782] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:45.221728 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba124fe8-5bed-4f75-8add-9b85d11c1a2a admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:45.225358 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ba124fe8-5bed-4f75-8add-9b85d11c1a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:45.430046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba124fe8-5bed-4f75-8add-9b85d11c1a2a admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:45.438705 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79e45b-aec1-4247-8e0b-d027449a75de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:45.440432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79e45b-aec1-4247-8e0b-d027449a75de service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:45.441471 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79e45b-aec1-4247-8e0b-d027449a75de service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:45.442518 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79e45b-aec1-4247-8e0b-d027449a75de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:45.474168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e79e45b-aec1-4247-8e0b-d027449a75de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:45.483433 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e79e45b-aec1-4247-8e0b-d027449a75de service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:45.513444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 888/1783] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:46:48.765317 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027ecbd-abec-4852-bce0-e6786f71c2db None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:48.767294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e027ecbd-abec-4852-bce0-e6786f71c2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:48.843446 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027ecbd-abec-4852-bce0-e6786f71c2db None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:48.878896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f932024e-3d34-4b05-92de-0afa5c99b13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:48.882880 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932024e-3d34-4b05-92de-0afa5c99b13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:48.883371 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932024e-3d34-4b05-92de-0afa5c99b13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:48.883919 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932024e-3d34-4b05-92de-0afa5c99b13f admin admin] PATH_INFO: `/v3/users/72cac493db1642acb9e367e045624f03` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:48.889451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f932024e-3d34-4b05-92de-0afa5c99b13f admin admin] RBAC: Authorizing `identity:delete_user(user_id=72cac493db1642acb9e367e045624f03)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:48.893388 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f932024e-3d34-4b05-92de-0afa5c99b13f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:49.325719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f932024e-3d34-4b05-92de-0afa5c99b13f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72cac493db1642acb9e367e045624f03'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:49.325719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f932024e-3d34-4b05-92de-0afa5c99b13f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72cac493db1642acb9e367e045624f03'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:49.325719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f932024e-3d34-4b05-92de-0afa5c99b13f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72cac493db1642acb9e367e045624f03'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:49.395528 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f932024e-3d34-4b05-92de-0afa5c99b13f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72cac493db1642acb9e367e045624f03'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:49.427966 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f932024e-3d34-4b05-92de-0afa5c99b13f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72cac493db1642acb9e367e045624f03'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:49.548961 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 896/1784] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:46:48 2022] DELETE /identity/v3/users/72cac493db1642acb9e367e045624f03 => generated 0 bytes in 786 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:49.557151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072861ee-f7d6-471c-837f-b7af3f2329cb None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:49.558643 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072861ee-f7d6-471c-837f-b7af3f2329cb None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:49.561516 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072861ee-f7d6-471c-837f-b7af3f2329cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:49.618135 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-072861ee-f7d6-471c-837f-b7af3f2329cb None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:49.619713 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-072861ee-f7d6-471c-837f-b7af3f2329cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:49.833270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 889/1785] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:46:49.900124 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f932024e-3d34-4b05-92de-0afa5c99b13f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:49.902244 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f932024e-3d34-4b05-92de-0afa5c99b13f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:49.908477 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072861ee-f7d6-471c-837f-b7af3f2329cb None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:49.911124 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-072861ee-f7d6-471c-837f-b7af3f2329cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:49.949248 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f932024e-3d34-4b05-92de-0afa5c99b13f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:49.960145 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b961952-6d23-4a90-9b77-23c1eb3c676e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:49.961720 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b961952-6d23-4a90-9b77-23c1eb3c676e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:49.962368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b961952-6d23-4a90-9b77-23c1eb3c676e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:49.962849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b961952-6d23-4a90-9b77-23c1eb3c676e admin admin] PATH_INFO: `/v3/projects/e1a369037c7c429ebd67e30059035364` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:49.965614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b961952-6d23-4a90-9b77-23c1eb3c676e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1a369037c7c429ebd67e30059035364)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:49.969401 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b961952-6d23-4a90-9b77-23c1eb3c676e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:50.025853 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:50.232752 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072861ee-f7d6-471c-837f-b7af3f2329cb None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:50.258093 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6b961952-6d23-4a90-9b77-23c1eb3c676e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1a369037c7c429ebd67e30059035364', 'request_id': 'req-6b961952-6d23-4a90-9b77-23c1eb3c676e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:50.299929 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6b961952-6d23-4a90-9b77-23c1eb3c676e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1a369037c7c429ebd67e30059035364', 'request_id': 'req-6b961952-6d23-4a90-9b77-23c1eb3c676e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:50.344851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6b961952-6d23-4a90-9b77-23c1eb3c676e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1a369037c7c429ebd67e30059035364', 'request_id': 'req-6b961952-6d23-4a90-9b77-23c1eb3c676e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:50.394793 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c837e06-a44f-47d8-86e9-8d5130ac8d32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:50.395999 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c837e06-a44f-47d8-86e9-8d5130ac8d32 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:50.396755 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c837e06-a44f-47d8-86e9-8d5130ac8d32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:50.397108 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c837e06-a44f-47d8-86e9-8d5130ac8d32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:50.412728 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:50.483855 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6b961952-6d23-4a90-9b77-23c1eb3c676e admin admin] The token cache is being invalidate because project e1a369037c7c429ebd67e30059035364 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:46:50.484806 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6b961952-6d23-4a90-9b77-23c1eb3c676e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:50.486267 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6b961952-6d23-4a90-9b77-23c1eb3c676e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:50.489999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 897/1786] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:46:49 2022] DELETE /identity/v3/projects/e1a369037c7c429ebd67e30059035364 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:50.497803 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b961952-6d23-4a90-9b77-23c1eb3c676e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:50.502842 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6b961952-6d23-4a90-9b77-23c1eb3c676e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:50.568664 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c837e06-a44f-47d8-86e9-8d5130ac8d32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:50.578974 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c837e06-a44f-47d8-86e9-8d5130ac8d32 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:50.645073 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 890/1787] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 738 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:46:50.801771 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b961952-6d23-4a90-9b77-23c1eb3c676e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:50.812664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9180c-214c-486f-afd6-30e76f6bca60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:50.814755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9180c-214c-486f-afd6-30e76f6bca60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:50.815560 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9180c-214c-486f-afd6-30e76f6bca60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:50.816678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9180c-214c-486f-afd6-30e76f6bca60 admin admin] PATH_INFO: `/v3/users/7263d7d21d5c4072a90126bf9262dedf` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:50.831439 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9180c-214c-486f-afd6-30e76f6bca60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7263d7d21d5c4072a90126bf9262dedf)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:50.836292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9180c-214c-486f-afd6-30e76f6bca60 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:51.119191 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b7e9180c-214c-486f-afd6-30e76f6bca60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7263d7d21d5c4072a90126bf9262dedf'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:51.119989 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b7e9180c-214c-486f-afd6-30e76f6bca60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7263d7d21d5c4072a90126bf9262dedf'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:51.121046 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b7e9180c-214c-486f-afd6-30e76f6bca60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7263d7d21d5c4072a90126bf9262dedf'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:51.179187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b7e9180c-214c-486f-afd6-30e76f6bca60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7263d7d21d5c4072a90126bf9262dedf'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:51.198814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b7e9180c-214c-486f-afd6-30e76f6bca60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7263d7d21d5c4072a90126bf9262dedf'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:51.299245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 898/1788] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:46:50 2022] DELETE /identity/v3/users/7263d7d21d5c4072a90126bf9262dedf => generated 0 bytes in 804 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:53.663712 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c837e06-a44f-47d8-86e9-8d5130ac8d32 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:53.678127 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c837e06-a44f-47d8-86e9-8d5130ac8d32 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:53.761370 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9180c-214c-486f-afd6-30e76f6bca60 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:53.763608 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b7e9180c-214c-486f-afd6-30e76f6bca60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:53.855088 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:53.885355 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e94abf-7a31-4ece-96a3-7304b3eb5a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:53.887091 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e94abf-7a31-4ece-96a3-7304b3eb5a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:53.890348 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e94abf-7a31-4ece-96a3-7304b3eb5a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:53.891079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e94abf-7a31-4ece-96a3-7304b3eb5a43 admin admin] PATH_INFO: `/v3/projects/1c5100b00c2144dc89f23d2ca98b4090` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:53.893185 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e94abf-7a31-4ece-96a3-7304b3eb5a43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c5100b00c2144dc89f23d2ca98b4090)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:53.896209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e94abf-7a31-4ece-96a3-7304b3eb5a43 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:53.923072 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9180c-214c-486f-afd6-30e76f6bca60 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:53.936959 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd5978-2968-46f6-bbfe-097d53d2a9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:53.938305 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd5978-2968-46f6-bbfe-097d53d2a9f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:53.939323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd5978-2968-46f6-bbfe-097d53d2a9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:53.939872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd5978-2968-46f6-bbfe-097d53d2a9f8 admin admin] PATH_INFO: `/v3/projects/48abe55177574cdb89c1e5725afc888b` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:53.942827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd5978-2968-46f6-bbfe-097d53d2a9f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48abe55177574cdb89c1e5725afc888b)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:53.946254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd5978-2968-46f6-bbfe-097d53d2a9f8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:54.038971 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a6e94abf-7a31-4ece-96a3-7304b3eb5a43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c5100b00c2144dc89f23d2ca98b4090', 'request_id': 'req-a6e94abf-7a31-4ece-96a3-7304b3eb5a43'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:54.048505 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a6e94abf-7a31-4ece-96a3-7304b3eb5a43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c5100b00c2144dc89f23d2ca98b4090', 'request_id': 'req-a6e94abf-7a31-4ece-96a3-7304b3eb5a43'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:54.057139 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a6e94abf-7a31-4ece-96a3-7304b3eb5a43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c5100b00c2144dc89f23d2ca98b4090', 'request_id': 'req-a6e94abf-7a31-4ece-96a3-7304b3eb5a43'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:54.108385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3edd5978-2968-46f6-bbfe-097d53d2a9f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48abe55177574cdb89c1e5725afc888b', 'request_id': 'req-3edd5978-2968-46f6-bbfe-097d53d2a9f8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:54.117659 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3edd5978-2968-46f6-bbfe-097d53d2a9f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48abe55177574cdb89c1e5725afc888b', 'request_id': 'req-3edd5978-2968-46f6-bbfe-097d53d2a9f8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:54.130870 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3edd5978-2968-46f6-bbfe-097d53d2a9f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48abe55177574cdb89c1e5725afc888b', 'request_id': 'req-3edd5978-2968-46f6-bbfe-097d53d2a9f8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:54.197225 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a6e94abf-7a31-4ece-96a3-7304b3eb5a43 admin admin] The token cache is being invalidate because project 1c5100b00c2144dc89f23d2ca98b4090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:46:54.197659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a6e94abf-7a31-4ece-96a3-7304b3eb5a43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:54.198977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a6e94abf-7a31-4ece-96a3-7304b3eb5a43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:54.205913 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 891/1789] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:46:53 2022] DELETE /identity/v3/projects/1c5100b00c2144dc89f23d2ca98b4090 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:54.214025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121410b9-0d33-45bc-a7c5-9303062b6c76 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:54.215168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121410b9-0d33-45bc-a7c5-9303062b6c76 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:54.216539 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121410b9-0d33-45bc-a7c5-9303062b6c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:54.294771 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-121410b9-0d33-45bc-a7c5-9303062b6c76 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:54.300336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-121410b9-0d33-45bc-a7c5-9303062b6c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:54.325315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3edd5978-2968-46f6-bbfe-097d53d2a9f8 admin admin] The token cache is being invalidate because project 48abe55177574cdb89c1e5725afc888b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:46:54.332472 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3edd5978-2968-46f6-bbfe-097d53d2a9f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:54.347320 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3edd5978-2968-46f6-bbfe-097d53d2a9f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:54.361108 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 899/1790] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:46:53 2022] DELETE /identity/v3/projects/48abe55177574cdb89c1e5725afc888b => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:54.363988 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd5978-2968-46f6-bbfe-097d53d2a9f8 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:54.389816 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3edd5978-2968-46f6-bbfe-097d53d2a9f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:54.438512 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:54.515543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 892/1791] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:46:54.516298 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21bc2e-084a-4492-94eb-4b02d991cccb None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:54.516587 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21bc2e-084a-4492-94eb-4b02d991cccb None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:54.516868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21bc2e-084a-4492-94eb-4b02d991cccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:54.551617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-bc21bc2e-084a-4492-94eb-4b02d991cccb None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:54.553208 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bc21bc2e-084a-4492-94eb-4b02d991cccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:54.579581 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 893/1792] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:46:54.583851 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21bc2e-084a-4492-94eb-4b02d991cccb None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:54.589499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd5978-2968-46f6-bbfe-097d53d2a9f8 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:54.601770 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21bc2e-084a-4492-94eb-4b02d991cccb None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:54.605616 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba1201-f0ef-4751-8d12-5118e3b53cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:54.607040 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba1201-f0ef-4751-8d12-5118e3b53cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:54.607581 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba1201-f0ef-4751-8d12-5118e3b53cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:54.608041 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba1201-f0ef-4751-8d12-5118e3b53cb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:54.608211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849346eb-1cc8-45ee-84ce-93e249d0ff5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:54.609036 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849346eb-1cc8-45ee-84ce-93e249d0ff5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:54.609234 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba1201-f0ef-4751-8d12-5118e3b53cb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:46:54.609389 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849346eb-1cc8-45ee-84ce-93e249d0ff5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:54.609555 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba1201-f0ef-4751-8d12-5118e3b53cb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:54.609714 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849346eb-1cc8-45ee-84ce-93e249d0ff5d admin admin] PATH_INFO: `/v3/users/0349a014e40f422bb1b5a6150f60cf60` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:54.611502 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba1201-f0ef-4751-8d12-5118e3b53cb7 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:54.620701 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849346eb-1cc8-45ee-84ce-93e249d0ff5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0349a014e40f422bb1b5a6150f60cf60)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:54.623422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849346eb-1cc8-45ee-84ce-93e249d0ff5d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:54.628644 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 894/1793] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:46:54.633425 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba1201-f0ef-4751-8d12-5118e3b53cb7 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:54.663039 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba1201-f0ef-4751-8d12-5118e3b53cb7 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:54.736174 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e2db7-e01c-4619-b688-170d971c5666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:54.741020 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e2db7-e01c-4619-b688-170d971c5666 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:54.744691 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e2db7-e01c-4619-b688-170d971c5666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:54.748438 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e2db7-e01c-4619-b688-170d971c5666 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:54.750312 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e2db7-e01c-4619-b688-170d971c5666 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:54.753515 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e2db7-e01c-4619-b688-170d971c5666 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:54.937626 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 895/1794] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:46:54 2022] POST /identity/v3/projects => generated 379 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:46:54.942525 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e2db7-e01c-4619-b688-170d971c5666 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:55.011995 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-849346eb-1cc8-45ee-84ce-93e249d0ff5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0349a014e40f422bb1b5a6150f60cf60'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:55.012912 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-849346eb-1cc8-45ee-84ce-93e249d0ff5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0349a014e40f422bb1b5a6150f60cf60'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:55.019528 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-849346eb-1cc8-45ee-84ce-93e249d0ff5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0349a014e40f422bb1b5a6150f60cf60'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:55.022660 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e2db7-e01c-4619-b688-170d971c5666 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:55.029774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366506ff-c265-4be9-8acb-94d07e171f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:55.030935 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366506ff-c265-4be9-8acb-94d07e171f63 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:55.031542 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366506ff-c265-4be9-8acb-94d07e171f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:55.032028 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366506ff-c265-4be9-8acb-94d07e171f63 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:55.033549 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366506ff-c265-4be9-8acb-94d07e171f63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:55.038222 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366506ff-c265-4be9-8acb-94d07e171f63 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:55.068626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-849346eb-1cc8-45ee-84ce-93e249d0ff5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0349a014e40f422bb1b5a6150f60cf60'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:55.088945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-849346eb-1cc8-45ee-84ce-93e249d0ff5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0349a014e40f422bb1b5a6150f60cf60'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:55.169399 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 896/1795] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:46:54 2022] POST /identity/v3/users => generated 290 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:46:55.170395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366506ff-c265-4be9-8acb-94d07e171f63 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:55.176081 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-366506ff-c265-4be9-8acb-94d07e171f63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:55.232056 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 900/1796] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:46:54 2022] DELETE /identity/v3/users/0349a014e40f422bb1b5a6150f60cf60 => generated 0 bytes in 866 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:55.237895 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366506ff-c265-4be9-8acb-94d07e171f63 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:55.238957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa2583-c6c0-4b6d-971b-54940124a34f None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:55.239410 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa2583-c6c0-4b6d-971b-54940124a34f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:55.239876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa2583-c6c0-4b6d-971b-54940124a34f None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:55.242115 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 901/1797] 10.222.0.57 () {54 vars in 921 bytes} [Mon Jul 18 08:46:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:46:55.250217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d775-9730-4741-998a-7babe39bc178 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:55.251597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d775-9730-4741-998a-7babe39bc178 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:55.251597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d775-9730-4741-998a-7babe39bc178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:55.289395 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-9792d775-9730-4741-998a-7babe39bc178 None None] MFA Rules not processed for user `84cba474fc7c470ea119de697b9d7ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:55.290945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9792d775-9730-4741-998a-7babe39bc178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:55.419954 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d62b12-33ca-4b8c-a0cc-a5aa419ee2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:55.420575 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d62b12-33ca-4b8c-a0cc-a5aa419ee2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:55.424386 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d62b12-33ca-4b8c-a0cc-a5aa419ee2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:55.433092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 902/1798] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Jul 18 08:46:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:55.441574 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c326e5-b1f3-4345-b6e2-718ad25e6ce2 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:55.441574 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c326e5-b1f3-4345-b6e2-718ad25e6ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:55.441574 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c326e5-b1f3-4345-b6e2-718ad25e6ce2 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:55.441574 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 903/1799] 10.222.0.57 () {54 vars in 921 bytes} [Mon Jul 18 08:46:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:46:55.443550 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d62b12-33ca-4b8c-a0cc-a5aa419ee2a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:55.447407 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d62b12-33ca-4b8c-a0cc-a5aa419ee2a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:55.478668 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d62b12-33ca-4b8c-a0cc-a5aa419ee2a2 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:55.524540 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:55.573388 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216964a-83e9-4fb5-8ff5-8918f734ffd8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:55.573388 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216964a-83e9-4fb5-8ff5-8918f734ffd8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:55.573388 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216964a-83e9-4fb5-8ff5-8918f734ffd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:55.588940 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 897/1800] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:55.593340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d62b12-33ca-4b8c-a0cc-a5aa419ee2a2 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:55.652859 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d62b12-33ca-4b8c-a0cc-a5aa419ee2a2 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:55.666830 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-6216964a-83e9-4fb5-8ff5-8918f734ffd8 None None] MFA Rules not processed for user `84cba474fc7c470ea119de697b9d7ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:55.673145 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6216964a-83e9-4fb5-8ff5-8918f734ffd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:55.748557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 904/1801] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Jul 18 08:46:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 18 08:46:55.753661 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6873b1e-10d3-4bdc-a327-42b7829a0022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:55.757173 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6873b1e-10d3-4bdc-a327-42b7829a0022 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:55.762872 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6873b1e-10d3-4bdc-a327-42b7829a0022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:55.763418 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6873b1e-10d3-4bdc-a327-42b7829a0022 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:55.786378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6873b1e-10d3-4bdc-a327-42b7829a0022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:55.804901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6873b1e-10d3-4bdc-a327-42b7829a0022 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:55.898932 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216964a-83e9-4fb5-8ff5-8918f734ffd8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:55.902398 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6216964a-83e9-4fb5-8ff5-8918f734ffd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:55.943868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 898/1802] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:55.945516 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6873b1e-10d3-4bdc-a327-42b7829a0022 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:56.008613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6873b1e-10d3-4bdc-a327-42b7829a0022 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:56.020081 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216964a-83e9-4fb5-8ff5-8918f734ffd8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:56.033748 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992f6d4-1400-4f76-bfdb-55107d359403 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:56.035344 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992f6d4-1400-4f76-bfdb-55107d359403 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:56.036186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992f6d4-1400-4f76-bfdb-55107d359403 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:56.037110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992f6d4-1400-4f76-bfdb-55107d359403 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:56.037541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc932c4-03eb-47a0-a24c-f19ec5489be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:56.039187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc932c4-03eb-47a0-a24c-f19ec5489be3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:56.039756 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc932c4-03eb-47a0-a24c-f19ec5489be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:56.040230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc932c4-03eb-47a0-a24c-f19ec5489be3 admin admin] PATH_INFO: `/v3/projects/58fbce56fe224c7ea295dc7d243efa2f/users/858f5d84447b4d218ccd313b4afdd555/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:56.065328 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0992f6d4-1400-4f76-bfdb-55107d359403 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:56.069053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0992f6d4-1400-4f76-bfdb-55107d359403 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:56.100547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc932c4-03eb-47a0-a24c-f19ec5489be3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58fbce56fe224c7ea295dc7d243efa2f, user_id=858f5d84447b4d218ccd313b4afdd555, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:56.104636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 905/1803] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:46:56.109000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc932c4-03eb-47a0-a24c-f19ec5489be3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:56.487837 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:56.506992 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 899/1804] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:46:55 2022] PUT /identity/v3/projects/58fbce56fe224c7ea295dc7d243efa2f/users/858f5d84447b4d218ccd313b4afdd555/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:56.529200 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b72b11-ff48-4a59-aa57-41e9448fdf8d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:56.530091 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b72b11-ff48-4a59-aa57-41e9448fdf8d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:56.530831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b72b11-ff48-4a59-aa57-41e9448fdf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:56.657069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-34b72b11-ff48-4a59-aa57-41e9448fdf8d None None] MFA Rules not processed for user `858f5d84447b4d218ccd313b4afdd555`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:56.661767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc932c4-03eb-47a0-a24c-f19ec5489be3 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:56.661939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-34b72b11-ff48-4a59-aa57-41e9448fdf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:56.662062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6fc932c4-03eb-47a0-a24c-f19ec5489be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:56.818499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 906/1805] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:56.862174 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b72b11-ff48-4a59-aa57-41e9448fdf8d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:56.967472 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b72b11-ff48-4a59-aa57-41e9448fdf8d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:56.977542 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9eead8-520b-4cf5-8a3b-430c35e885d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:56.979174 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9eead8-520b-4cf5-8a3b-430c35e885d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:56.980219 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9eead8-520b-4cf5-8a3b-430c35e885d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:56.980996 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9eead8-520b-4cf5-8a3b-430c35e885d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:56.982949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9eead8-520b-4cf5-8a3b-430c35e885d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:56.987616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9eead8-520b-4cf5-8a3b-430c35e885d0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:57.011996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc932c4-03eb-47a0-a24c-f19ec5489be3 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:57.024846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6582bc-aa5f-46af-9d19-bc025eb0f1a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:57.026026 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6582bc-aa5f-46af-9d19-bc025eb0f1a1 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:57.026528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6582bc-aa5f-46af-9d19-bc025eb0f1a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:57.027032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6582bc-aa5f-46af-9d19-bc025eb0f1a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:57.045391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6582bc-aa5f-46af-9d19-bc025eb0f1a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:57.048630 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6582bc-aa5f-46af-9d19-bc025eb0f1a1 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:57.071066 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 900/1806] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:46:57.189568 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 907/1807] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:46:56 2022] POST /identity/v3/projects => generated 379 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:46:57.191132 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6582bc-aa5f-46af-9d19-bc025eb0f1a1 service nova] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:57.234580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6582bc-aa5f-46af-9d19-bc025eb0f1a1 service nova] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:57.393408 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9eead8-520b-4cf5-8a3b-430c35e885d0 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:57.399307 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0f9eead8-520b-4cf5-8a3b-430c35e885d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:57.437885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c82623-667a-4654-a9d4-0a002ffe6979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:57.439419 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c82623-667a-4654-a9d4-0a002ffe6979 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:57.440068 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c82623-667a-4654-a9d4-0a002ffe6979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:57.441104 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c82623-667a-4654-a9d4-0a002ffe6979 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:57.442734 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c82623-667a-4654-a9d4-0a002ffe6979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:57.447874 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c82623-667a-4654-a9d4-0a002ffe6979 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:57.583476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 901/1808] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:46:57 2022] POST /identity/v3/users => generated 290 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:46:57.586032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c82623-667a-4654-a9d4-0a002ffe6979 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:57.600426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c82623-667a-4654-a9d4-0a002ffe6979 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:57.609264 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9eead8-520b-4cf5-8a3b-430c35e885d0 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:57.611779 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdcc6a1-f6d4-4031-b8d9-983a7e5ec029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:57.614432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcc6a1-f6d4-4031-b8d9-983a7e5ec029 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:57.614973 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcc6a1-f6d4-4031-b8d9-983a7e5ec029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:57.615469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcc6a1-f6d4-4031-b8d9-983a7e5ec029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:57.616415 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdcc6a1-f6d4-4031-b8d9-983a7e5ec029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:57.618027 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9dd2b-a0eb-44eb-8c90-6d58e9ca0f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:57.619840 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9dd2b-a0eb-44eb-8c90-6d58e9ca0f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:57.620712 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9dd2b-a0eb-44eb-8c90-6d58e9ca0f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:57.621466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9dd2b-a0eb-44eb-8c90-6d58e9ca0f75 admin admin] PATH_INFO: `/v3/projects/b2320c44ac1c4954ab40fb192e9a4254` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:57.621967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdcc6a1-f6d4-4031-b8d9-983a7e5ec029 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:57.625071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad9dd2b-a0eb-44eb-8c90-6d58e9ca0f75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2320c44ac1c4954ab40fb192e9a4254)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:57.632539 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad9dd2b-a0eb-44eb-8c90-6d58e9ca0f75 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:57.651959 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 902/1809] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:57.656747 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdcc6a1-f6d4-4031-b8d9-983a7e5ec029 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:57.671094 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdcc6a1-f6d4-4031-b8d9-983a7e5ec029 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:57.677508 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9882b65-e2b5-4f37-a927-a69281b0800b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:57.678238 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9882b65-e2b5-4f37-a927-a69281b0800b admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:57.678536 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9882b65-e2b5-4f37-a927-a69281b0800b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:57.678840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9882b65-e2b5-4f37-a927-a69281b0800b admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:57.679698 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9882b65-e2b5-4f37-a927-a69281b0800b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:57.681700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9882b65-e2b5-4f37-a927-a69281b0800b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:57.696867 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 903/1810] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:57.703108 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9882b65-e2b5-4f37-a927-a69281b0800b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:57.725707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9882b65-e2b5-4f37-a927-a69281b0800b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:57.736524 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14860b0-3e4d-4a4b-8470-cbfcd5398b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:57.738968 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14860b0-3e4d-4a4b-8470-cbfcd5398b27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:57.740466 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14860b0-3e4d-4a4b-8470-cbfcd5398b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:57.741158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14860b0-3e4d-4a4b-8470-cbfcd5398b27 admin admin] PATH_INFO: `/v3/projects/318651762ce54d3b8664622c868e84f1/users/d2f9f3eae3e64be7aa90c2a097284c0c/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:57.775463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:57.779767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14860b0-3e4d-4a4b-8470-cbfcd5398b27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=318651762ce54d3b8664622c868e84f1, user_id=d2f9f3eae3e64be7aa90c2a097284c0c, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:57.786204 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14860b0-3e4d-4a4b-8470-cbfcd5398b27 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:57.940592 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 904/1811] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:46:57 2022] PUT /identity/v3/projects/318651762ce54d3b8664622c868e84f1/users/d2f9f3eae3e64be7aa90c2a097284c0c/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:57.948240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb368c-817f-49a2-ad07-188c435ddbb9 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:57.948847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb368c-817f-49a2-ad07-188c435ddbb9 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:57.949467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb368c-817f-49a2-ad07-188c435ddbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:57.958448 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cad9dd2b-a0eb-44eb-8c90-6d58e9ca0f75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2320c44ac1c4954ab40fb192e9a4254', 'request_id': 'req-cad9dd2b-a0eb-44eb-8c90-6d58e9ca0f75'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:57.976195 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cad9dd2b-a0eb-44eb-8c90-6d58e9ca0f75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2320c44ac1c4954ab40fb192e9a4254', 'request_id': 'req-cad9dd2b-a0eb-44eb-8c90-6d58e9ca0f75'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:57.996252 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cad9dd2b-a0eb-44eb-8c90-6d58e9ca0f75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2320c44ac1c4954ab40fb192e9a4254', 'request_id': 'req-cad9dd2b-a0eb-44eb-8c90-6d58e9ca0f75'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:58.100421 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:58.120005 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-eecb368c-817f-49a2-ad07-188c435ddbb9 None None] MFA Rules not processed for user `d2f9f3eae3e64be7aa90c2a097284c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:58.123010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-eecb368c-817f-49a2-ad07-188c435ddbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:58.124150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cad9dd2b-a0eb-44eb-8c90-6d58e9ca0f75 admin admin] The token cache is being invalidate because project b2320c44ac1c4954ab40fb192e9a4254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:46:58.124630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cad9dd2b-a0eb-44eb-8c90-6d58e9ca0f75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:58.128260 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cad9dd2b-a0eb-44eb-8c90-6d58e9ca0f75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:46:58.138234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 908/1812] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:46:57 2022] DELETE /identity/v3/projects/b2320c44ac1c4954ab40fb192e9a4254 => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:58.144619 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4252ea-c8bc-4468-8857-0be991fbc38e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:58.145064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4252ea-c8bc-4468-8857-0be991fbc38e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:58.145811 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4252ea-c8bc-4468-8857-0be991fbc38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:58.241680 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1d4252ea-c8bc-4468-8857-0be991fbc38e None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:58.244108 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1d4252ea-c8bc-4468-8857-0be991fbc38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:58.404033 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 905/1813] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:46:58.411249 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb368c-817f-49a2-ad07-188c435ddbb9 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:58.416250 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-eecb368c-817f-49a2-ad07-188c435ddbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:58.445092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 909/1814] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:46:58.453728 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb368c-817f-49a2-ad07-188c435ddbb9 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:58.456539 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281c277-4b11-47e9-85e7-1127e43ac5b6 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:58.456895 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281c277-4b11-47e9-85e7-1127e43ac5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:58.457187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281c277-4b11-47e9-85e7-1127e43ac5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:58.460159 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510b249-b42b-4412-92c6-d307c081da7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:58.460952 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510b249-b42b-4412-92c6-d307c081da7d admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:58.461305 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510b249-b42b-4412-92c6-d307c081da7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:58.461674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510b249-b42b-4412-92c6-d307c081da7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:58.463520 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510b249-b42b-4412-92c6-d307c081da7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:58.465946 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510b249-b42b-4412-92c6-d307c081da7d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:58.518027 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7281c277-4b11-47e9-85e7-1127e43ac5b6 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:58.524541 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7281c277-4b11-47e9-85e7-1127e43ac5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:58.547865 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 910/1815] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:58.553781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281c277-4b11-47e9-85e7-1127e43ac5b6 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:58.569272 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281c277-4b11-47e9-85e7-1127e43ac5b6 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:58.576402 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd822e30-1b6f-4adf-8a44-232fb15fde96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:58.585054 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd822e30-1b6f-4adf-8a44-232fb15fde96 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:58.585054 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd822e30-1b6f-4adf-8a44-232fb15fde96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:58.585054 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd822e30-1b6f-4adf-8a44-232fb15fde96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:58.585054 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd822e30-1b6f-4adf-8a44-232fb15fde96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:46:58.585054 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd822e30-1b6f-4adf-8a44-232fb15fde96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:58.585054 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd822e30-1b6f-4adf-8a44-232fb15fde96 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:58.613781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 911/1816] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:46:58.615510 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd822e30-1b6f-4adf-8a44-232fb15fde96 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:58.632510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 906/1817] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:46:58 2022] POST /identity/v3/projects => generated 379 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:46:58.633623 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510b249-b42b-4412-92c6-d307c081da7d admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:58.639515 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd822e30-1b6f-4adf-8a44-232fb15fde96 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:58.782822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510b249-b42b-4412-92c6-d307c081da7d admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:58.792584 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321b19d-2be3-4a9e-aa2a-0a6e9543c253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:58.795458 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321b19d-2be3-4a9e-aa2a-0a6e9543c253 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:58.796304 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321b19d-2be3-4a9e-aa2a-0a6e9543c253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:58.797030 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321b19d-2be3-4a9e-aa2a-0a6e9543c253 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:58.798602 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321b19d-2be3-4a9e-aa2a-0a6e9543c253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:58.798820 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7b0d0-e72d-43dc-a15c-16fa868f22e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:58.800193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7b0d0-e72d-43dc-a15c-16fa868f22e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:58.800650 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7b0d0-e72d-43dc-a15c-16fa868f22e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:58.801137 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7b0d0-e72d-43dc-a15c-16fa868f22e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:58.802506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7b0d0-e72d-43dc-a15c-16fa868f22e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:58.804103 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321b19d-2be3-4a9e-aa2a-0a6e9543c253 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:58.806709 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7b0d0-e72d-43dc-a15c-16fa868f22e7 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:58.960717 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 907/1818] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:46:58 2022] POST /identity/v3/users => generated 296 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:46:58.971225 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7b0d0-e72d-43dc-a15c-16fa868f22e7 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:59.011511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 912/1819] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:46:58 2022] POST /identity/v3/projects => generated 369 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:46:59.018102 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321b19d-2be3-4a9e-aa2a-0a6e9543c253 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:59.033059 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7b0d0-e72d-43dc-a15c-16fa868f22e7 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:59.191225 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321b19d-2be3-4a9e-aa2a-0a6e9543c253 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:59.202140 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9af80-5f73-4aa4-9b7a-10c3f09c8dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:59.203484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9af80-5f73-4aa4-9b7a-10c3f09c8dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:59.204015 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9af80-5f73-4aa4-9b7a-10c3f09c8dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:59.204494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9af80-5f73-4aa4-9b7a-10c3f09c8dc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:59.205842 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9af80-5f73-4aa4-9b7a-10c3f09c8dc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:59.210129 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9af80-5f73-4aa4-9b7a-10c3f09c8dc9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:59.220402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f17e88-cc6d-464a-ba58-6582001d6c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:59.221982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f17e88-cc6d-464a-ba58-6582001d6c64 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:59.224069 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f17e88-cc6d-464a-ba58-6582001d6c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:59.224069 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f17e88-cc6d-464a-ba58-6582001d6c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:59.235398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f17e88-cc6d-464a-ba58-6582001d6c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:59.235398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f17e88-cc6d-464a-ba58-6582001d6c64 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:59.253562 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 908/1820] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:59.255829 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f17e88-cc6d-464a-ba58-6582001d6c64 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:59.282745 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f17e88-cc6d-464a-ba58-6582001d6c64 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:59.296036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccea666-ab2f-45ce-9497-92c6357c8888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:59.299812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccea666-ab2f-45ce-9497-92c6357c8888 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:59.299812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccea666-ab2f-45ce-9497-92c6357c8888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:59.299812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccea666-ab2f-45ce-9497-92c6357c8888 admin admin] PATH_INFO: `/v3/projects/7827a6f853164504a1a2e5093615953a/users/e4851a3c98514217af3cb6a35f3961f9/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:59.330365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccea666-ab2f-45ce-9497-92c6357c8888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7827a6f853164504a1a2e5093615953a, user_id=e4851a3c98514217af3cb6a35f3961f9, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:59.333840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccea666-ab2f-45ce-9497-92c6357c8888 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:59.341485 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 913/1821] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:46:59 2022] POST /identity/v3/users => generated 285 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:46:59.342720 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9af80-5f73-4aa4-9b7a-10c3f09c8dc9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:59.354229 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9af80-5f73-4aa4-9b7a-10c3f09c8dc9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:59.360946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8657b-3fb0-4016-9740-3108fa97b117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:59.361835 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8657b-3fb0-4016-9740-3108fa97b117 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:59.362199 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8657b-3fb0-4016-9740-3108fa97b117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:59.362560 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8657b-3fb0-4016-9740-3108fa97b117 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:59.363463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba8657b-3fb0-4016-9740-3108fa97b117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:59.365516 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba8657b-3fb0-4016-9740-3108fa97b117 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:59.369558 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:59.379028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 914/1822] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:59.383030 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8657b-3fb0-4016-9740-3108fa97b117 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:59.399220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8657b-3fb0-4016-9740-3108fa97b117 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:59.407868 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94529fd-4f1b-4309-9ee5-be1240a028cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:59.409293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94529fd-4f1b-4309-9ee5-be1240a028cd admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:59.409293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94529fd-4f1b-4309-9ee5-be1240a028cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:59.409623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94529fd-4f1b-4309-9ee5-be1240a028cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:59.410660 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94529fd-4f1b-4309-9ee5-be1240a028cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:59.412697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94529fd-4f1b-4309-9ee5-be1240a028cd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:59.427351 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 915/1823] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:46:59.431977 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94529fd-4f1b-4309-9ee5-be1240a028cd admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:59.439497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 909/1824] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:46:59 2022] PUT /identity/v3/projects/7827a6f853164504a1a2e5093615953a/users/e4851a3c98514217af3cb6a35f3961f9/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:59.445255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94529fd-4f1b-4309-9ee5-be1240a028cd admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:59.445586 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccea666-ab2f-45ce-9497-92c6357c8888 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:59.525097 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ec57a-81f4-4768-b102-a9b01305898c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:59.526043 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ec57a-81f4-4768-b102-a9b01305898c admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:59.526410 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ec57a-81f4-4768-b102-a9b01305898c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:59.527180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ec57a-81f4-4768-b102-a9b01305898c admin admin] PATH_INFO: `/v3/projects/54a350b1089242b8864f9d5a50b0c7ba/users/84fee63b7c604c88838ee451218afa7f/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:59.546974 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ec57a-81f4-4768-b102-a9b01305898c admin admin] RBAC: Authorizing `identity:create_grant(project_id=54a350b1089242b8864f9d5a50b0c7ba, user_id=84fee63b7c604c88838ee451218afa7f, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:59.550103 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ec57a-81f4-4768-b102-a9b01305898c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:59.560453 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccea666-ab2f-45ce-9497-92c6357c8888 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:59.574105 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc3fe2-8c50-4426-8d88-3c078801c410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:59.575213 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc3fe2-8c50-4426-8d88-3c078801c410 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:59.575878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc3fe2-8c50-4426-8d88-3c078801c410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:59.576264 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc3fe2-8c50-4426-8d88-3c078801c410 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:59.577555 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc3fe2-8c50-4426-8d88-3c078801c410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:59.579863 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc3fe2-8c50-4426-8d88-3c078801c410 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:59.597819 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 910/1825] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:46:59.601402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc3fe2-8c50-4426-8d88-3c078801c410 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:59.618168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc3fe2-8c50-4426-8d88-3c078801c410 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:59.627259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00712448-9f7d-4dc7-ab10-659704291663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:59.628901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00712448-9f7d-4dc7-ab10-659704291663 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:59.629568 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00712448-9f7d-4dc7-ab10-659704291663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:59.630085 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00712448-9f7d-4dc7-ab10-659704291663 admin admin] PATH_INFO: `/v3/projects/7827a6f853164504a1a2e5093615953a/users/e4851a3c98514217af3cb6a35f3961f9/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:59.633130 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00712448-9f7d-4dc7-ab10-659704291663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7827a6f853164504a1a2e5093615953a, user_id=e4851a3c98514217af3cb6a35f3961f9, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:59.639150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00712448-9f7d-4dc7-ab10-659704291663 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:46:59.660425 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 916/1826] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:46:59 2022] PUT /identity/v3/projects/54a350b1089242b8864f9d5a50b0c7ba/users/84fee63b7c604c88838ee451218afa7f/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:46:59.665238 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 911/1827] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:46:59 2022] PUT /identity/v3/projects/7827a6f853164504a1a2e5093615953a/users/e4851a3c98514217af3cb6a35f3961f9/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:46:59.670958 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398a230-043f-45b4-a8c4-551a77af78ab None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:59.671491 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398a230-043f-45b4-a8c4-551a77af78ab None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:59.672039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398a230-043f-45b4-a8c4-551a77af78ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:59.683570 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420890a-0b85-4666-a50d-f73176c4b783 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:59.686720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420890a-0b85-4666-a50d-f73176c4b783 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:59.690621 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420890a-0b85-4666-a50d-f73176c4b783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:59.713212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f398a230-043f-45b4-a8c4-551a77af78ab None None] MFA Rules not processed for user `84fee63b7c604c88838ee451218afa7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:59.714471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f398a230-043f-45b4-a8c4-551a77af78ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:59.743340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6420890a-0b85-4666-a50d-f73176c4b783 None None] MFA Rules not processed for user `e4851a3c98514217af3cb6a35f3961f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:46:59.745199 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6420890a-0b85-4666-a50d-f73176c4b783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:59.754166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:46:59.816492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 917/1828] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:59 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:59.817531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398a230-043f-45b4-a8c4-551a77af78ab None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:59.871224 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 912/1829] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:46:59 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:46:59.893738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420890a-0b85-4666-a50d-f73176c4b783 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:46:59.897646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6420890a-0b85-4666-a50d-f73176c4b783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:46:59.939459 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398a230-043f-45b4-a8c4-551a77af78ab None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:46:59.945420 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc6164-4b55-40da-b73a-db4781a52d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:46:59.949398 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc6164-4b55-40da-b73a-db4781a52d20 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:46:59.955237 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc6164-4b55-40da-b73a-db4781a52d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:46:59.955237 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc6164-4b55-40da-b73a-db4781a52d20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:46:59.955237 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc6164-4b55-40da-b73a-db4781a52d20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:46:59.957699 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc6164-4b55-40da-b73a-db4781a52d20 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:00.011554 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420890a-0b85-4666-a50d-f73176c4b783 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:00.016978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4c0b5-d538-43d1-a761-d4199b65e9bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:00.018997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4c0b5-d538-43d1-a761-d4199b65e9bf service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:00.018997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4c0b5-d538-43d1-a761-d4199b65e9bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:00.018997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4c0b5-d538-43d1-a761-d4199b65e9bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:00.035878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b4c0b5-d538-43d1-a761-d4199b65e9bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:00.037554 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b4c0b5-d538-43d1-a761-d4199b65e9bf service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:00.050832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 913/1830] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:46:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:47:00.095492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 918/1831] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:46:59 2022] POST /identity/v3/projects => generated 369 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:47:00.101698 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4c0b5-d538-43d1-a761-d4199b65e9bf service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:00.122977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4c0b5-d538-43d1-a761-d4199b65e9bf service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:00.238434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc6164-4b55-40da-b73a-db4781a52d20 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:00.241069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f3cc6164-4b55-40da-b73a-db4781a52d20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:00.246163 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd2e3c-3192-48f9-9d54-f8f615fdb067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:00.248297 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd2e3c-3192-48f9-9d54-f8f615fdb067 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:00.248855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd2e3c-3192-48f9-9d54-f8f615fdb067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:00.249399 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd2e3c-3192-48f9-9d54-f8f615fdb067 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:00.250789 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd2e3c-3192-48f9-9d54-f8f615fdb067 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:00.255033 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd2e3c-3192-48f9-9d54-f8f615fdb067 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:00.336437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc6164-4b55-40da-b73a-db4781a52d20 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:00.341430 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b3d1e-9a75-427e-af7b-edee45459d89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:00.342679 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b3d1e-9a75-427e-af7b-edee45459d89 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:00.343776 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b3d1e-9a75-427e-af7b-edee45459d89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:00.343776 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b3d1e-9a75-427e-af7b-edee45459d89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:00.348934 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-361b3d1e-9a75-427e-af7b-edee45459d89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:00.395806 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 914/1832] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:47:00 2022] POST /identity/v3/users => generated 291 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:00.398494 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd2e3c-3192-48f9-9d54-f8f615fdb067 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:00.418305 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd2e3c-3192-48f9-9d54-f8f615fdb067 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:00.435348 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa395acd-4b48-4770-bf12-38919b706339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:00.438983 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa395acd-4b48-4770-bf12-38919b706339 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:00.440107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa395acd-4b48-4770-bf12-38919b706339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:00.441406 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa395acd-4b48-4770-bf12-38919b706339 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:00.453117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa395acd-4b48-4770-bf12-38919b706339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:00.453117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa395acd-4b48-4770-bf12-38919b706339 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:00.475565 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:00.482376 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b3d1e-9a75-427e-af7b-edee45459d89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:00.482376 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b3d1e-9a75-427e-af7b-edee45459d89 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:00.485709 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 915/1833] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:00.491730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa395acd-4b48-4770-bf12-38919b706339 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:00.508168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 919/1834] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:47:00.524410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa395acd-4b48-4770-bf12-38919b706339 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:00.537304 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1e89e-0180-4aa8-949b-0320a2bddf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:00.538359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1e89e-0180-4aa8-949b-0320a2bddf35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:00.538647 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1e89e-0180-4aa8-949b-0320a2bddf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:00.538920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1e89e-0180-4aa8-949b-0320a2bddf35 admin admin] PATH_INFO: `/v3/projects/edd96bfbf463468d90645f74d2397c5c/users/c7e0e39a501640828863644d19a55e02/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:00.576280 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1e89e-0180-4aa8-949b-0320a2bddf35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edd96bfbf463468d90645f74d2397c5c, user_id=c7e0e39a501640828863644d19a55e02, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:00.579420 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1e89e-0180-4aa8-949b-0320a2bddf35 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:00.678905 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 916/1835] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:00 2022] PUT /identity/v3/projects/edd96bfbf463468d90645f74d2397c5c/users/c7e0e39a501640828863644d19a55e02/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:00.686443 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b3d1e-9a75-427e-af7b-edee45459d89 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:00.701831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b3d1e-9a75-427e-af7b-edee45459d89 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:00.833454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a54147a-5b70-42ad-9f0d-7fb1a5698b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:00.834334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54147a-5b70-42ad-9f0d-7fb1a5698b6d admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:00.835005 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54147a-5b70-42ad-9f0d-7fb1a5698b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:00.835457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54147a-5b70-42ad-9f0d-7fb1a5698b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:00.836465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a54147a-5b70-42ad-9f0d-7fb1a5698b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:00.839567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a54147a-5b70-42ad-9f0d-7fb1a5698b6d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:00.854854 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 920/1836] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:00.862543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1e89e-0180-4aa8-949b-0320a2bddf35 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:00.880947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1e89e-0180-4aa8-949b-0320a2bddf35 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:00.888348 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084f655-91c4-40ce-8e8b-1505ec574f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:00.889595 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084f655-91c4-40ce-8e8b-1505ec574f19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:00.890020 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084f655-91c4-40ce-8e8b-1505ec574f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:00.890402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084f655-91c4-40ce-8e8b-1505ec574f19 admin admin] PATH_INFO: `/v3/projects/edd96bfbf463468d90645f74d2397c5c/users/c7e0e39a501640828863644d19a55e02/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:00.890572 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:00.893893 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084f655-91c4-40ce-8e8b-1505ec574f19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edd96bfbf463468d90645f74d2397c5c, user_id=c7e0e39a501640828863644d19a55e02, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:00.898073 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084f655-91c4-40ce-8e8b-1505ec574f19 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:00.912620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 917/1837] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:00 2022] PUT /identity/v3/projects/edd96bfbf463468d90645f74d2397c5c/users/c7e0e39a501640828863644d19a55e02/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:00.932713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca61c7b-5b8d-424d-aad6-4574625903ec None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:00.933513 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca61c7b-5b8d-424d-aad6-4574625903ec None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:00.933891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca61c7b-5b8d-424d-aad6-4574625903ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:01.003285 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-5ca61c7b-5b8d-424d-aad6-4574625903ec None None] MFA Rules not processed for user `c7e0e39a501640828863644d19a55e02`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:01.005118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5ca61c7b-5b8d-424d-aad6-4574625903ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:01.090188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:01.160607 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 921/1838] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:00 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:01.206092 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084f655-91c4-40ce-8e8b-1505ec574f19 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:01.296780 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:01.303463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084f655-91c4-40ce-8e8b-1505ec574f19 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:01.308072 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86188374-cec3-420d-9c87-50a0bccf0d06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:01.308840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86188374-cec3-420d-9c87-50a0bccf0d06 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:01.309125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86188374-cec3-420d-9c87-50a0bccf0d06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:01.310176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86188374-cec3-420d-9c87-50a0bccf0d06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:01.325056 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86188374-cec3-420d-9c87-50a0bccf0d06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:01.334223 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86188374-cec3-420d-9c87-50a0bccf0d06 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:01.347068 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 918/1839] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:01 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:47:01.744399 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca61c7b-5b8d-424d-aad6-4574625903ec None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:01.757410 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca61c7b-5b8d-424d-aad6-4574625903ec None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:01.762670 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c74431-0567-4ebc-9b65-0ee573f73b24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:01.764618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c74431-0567-4ebc-9b65-0ee573f73b24 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:01.765161 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c74431-0567-4ebc-9b65-0ee573f73b24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:01.765671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c74431-0567-4ebc-9b65-0ee573f73b24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:01.873064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c74431-0567-4ebc-9b65-0ee573f73b24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:01.876232 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c74431-0567-4ebc-9b65-0ee573f73b24 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:01.897190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 922/1840] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:01 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:47:06.264488 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86188374-cec3-420d-9c87-50a0bccf0d06 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:06.340572 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:06.426083 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86188374-cec3-420d-9c87-50a0bccf0d06 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:06.437599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d4c48-49ca-4787-9a36-9c073a6a555b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:06.439489 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d4c48-49ca-4787-9a36-9c073a6a555b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:06.440055 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d4c48-49ca-4787-9a36-9c073a6a555b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:06.440580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d4c48-49ca-4787-9a36-9c073a6a555b admin admin] PATH_INFO: `/v3/users/858f5d84447b4d218ccd313b4afdd555` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:06.443027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d4c48-49ca-4787-9a36-9c073a6a555b admin admin] RBAC: Authorizing `identity:delete_user(user_id=858f5d84447b4d218ccd313b4afdd555)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:06.445719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d4c48-49ca-4787-9a36-9c073a6a555b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:06.728907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d36d4c48-49ca-4787-9a36-9c073a6a555b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '858f5d84447b4d218ccd313b4afdd555'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:06.730236 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d36d4c48-49ca-4787-9a36-9c073a6a555b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '858f5d84447b4d218ccd313b4afdd555'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:06.732434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d36d4c48-49ca-4787-9a36-9c073a6a555b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '858f5d84447b4d218ccd313b4afdd555'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:06.755308 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d36d4c48-49ca-4787-9a36-9c073a6a555b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '858f5d84447b4d218ccd313b4afdd555'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:06.766683 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-d36d4c48-49ca-4787-9a36-9c073a6a555b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '858f5d84447b4d218ccd313b4afdd555'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:06.890950 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 919/1841] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:06 2022] DELETE /identity/v3/users/858f5d84447b4d218ccd313b4afdd555 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:06.897870 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527d90f-e35b-4cbe-8a4c-14991373c71e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:06.900235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527d90f-e35b-4cbe-8a4c-14991373c71e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:06.901106 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527d90f-e35b-4cbe-8a4c-14991373c71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:06.942050 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3527d90f-e35b-4cbe-8a4c-14991373c71e None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:06.944559 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3527d90f-e35b-4cbe-8a4c-14991373c71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:07.037450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 923/1842] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:07.051706 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d4c48-49ca-4787-9a36-9c073a6a555b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:07.053494 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d36d4c48-49ca-4787-9a36-9c073a6a555b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:07.143334 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d4c48-49ca-4787-9a36-9c073a6a555b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:07.159963 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ee7fa-8091-46c4-80f2-6ac93ad1e708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:07.159963 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ee7fa-8091-46c4-80f2-6ac93ad1e708 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:07.159963 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ee7fa-8091-46c4-80f2-6ac93ad1e708 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:07.159963 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ee7fa-8091-46c4-80f2-6ac93ad1e708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:07.164117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ee7fa-8091-46c4-80f2-6ac93ad1e708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:07.166593 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ee7fa-8091-46c4-80f2-6ac93ad1e708 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:07.181185 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 920/1843] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:47:07.926823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527d90f-e35b-4cbe-8a4c-14991373c71e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:07.932935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3527d90f-e35b-4cbe-8a4c-14991373c71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:07.965282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527d90f-e35b-4cbe-8a4c-14991373c71e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:07.973645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f61a8-da83-46de-8ae7-500679ccd3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:07.975793 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f61a8-da83-46de-8ae7-500679ccd3f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:07.976448 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f61a8-da83-46de-8ae7-500679ccd3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:07.976921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f61a8-da83-46de-8ae7-500679ccd3f8 admin admin] PATH_INFO: `/v3/projects/58fbce56fe224c7ea295dc7d243efa2f` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:07.979277 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f61a8-da83-46de-8ae7-500679ccd3f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58fbce56fe224c7ea295dc7d243efa2f)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:07.982959 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f61a8-da83-46de-8ae7-500679ccd3f8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:08.137411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4e4f61a8-da83-46de-8ae7-500679ccd3f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58fbce56fe224c7ea295dc7d243efa2f', 'request_id': 'req-4e4f61a8-da83-46de-8ae7-500679ccd3f8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:08.150089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4e4f61a8-da83-46de-8ae7-500679ccd3f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58fbce56fe224c7ea295dc7d243efa2f', 'request_id': 'req-4e4f61a8-da83-46de-8ae7-500679ccd3f8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:08.160546 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4e4f61a8-da83-46de-8ae7-500679ccd3f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58fbce56fe224c7ea295dc7d243efa2f', 'request_id': 'req-4e4f61a8-da83-46de-8ae7-500679ccd3f8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:08.272499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4e4f61a8-da83-46de-8ae7-500679ccd3f8 admin admin] The token cache is being invalidate because project 58fbce56fe224c7ea295dc7d243efa2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:08.273083 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4e4f61a8-da83-46de-8ae7-500679ccd3f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:08.274282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4e4f61a8-da83-46de-8ae7-500679ccd3f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:08.277257 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 924/1844] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:07 2022] DELETE /identity/v3/projects/58fbce56fe224c7ea295dc7d243efa2f => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:08.283523 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ee7fa-8091-46c4-80f2-6ac93ad1e708 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:08.308402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ee7fa-8091-46c4-80f2-6ac93ad1e708 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:08.447225 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b6134-603c-49ec-9b11-4a15b8865ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:08.449307 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b6134-603c-49ec-9b11-4a15b8865ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:08.449907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b6134-603c-49ec-9b11-4a15b8865ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:08.450569 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b6134-603c-49ec-9b11-4a15b8865ebd admin admin] PATH_INFO: `/v3/users/d2f9f3eae3e64be7aa90c2a097284c0c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:08.453499 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b6134-603c-49ec-9b11-4a15b8865ebd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2f9f3eae3e64be7aa90c2a097284c0c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:08.458414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b6134-603c-49ec-9b11-4a15b8865ebd admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:08.462124 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:08.768269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8e8b6134-603c-49ec-9b11-4a15b8865ebd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2f9f3eae3e64be7aa90c2a097284c0c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:08.769011 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8e8b6134-603c-49ec-9b11-4a15b8865ebd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2f9f3eae3e64be7aa90c2a097284c0c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:08.769661 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8e8b6134-603c-49ec-9b11-4a15b8865ebd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2f9f3eae3e64be7aa90c2a097284c0c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:08.787235 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8e8b6134-603c-49ec-9b11-4a15b8865ebd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2f9f3eae3e64be7aa90c2a097284c0c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:08.801060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8e8b6134-603c-49ec-9b11-4a15b8865ebd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2f9f3eae3e64be7aa90c2a097284c0c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:08.921236 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 921/1845] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:08 2022] DELETE /identity/v3/users/d2f9f3eae3e64be7aa90c2a097284c0c => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:09.156482 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f61a8-da83-46de-8ae7-500679ccd3f8 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:09.158304 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4e4f61a8-da83-46de-8ae7-500679ccd3f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:09.173912 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f61a8-da83-46de-8ae7-500679ccd3f8 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:09.198301 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:09.251951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47372f03-8c26-4f63-9a8e-9c754f67dae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:09.251951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47372f03-8c26-4f63-9a8e-9c754f67dae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:09.251951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47372f03-8c26-4f63-9a8e-9c754f67dae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:09.251951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47372f03-8c26-4f63-9a8e-9c754f67dae3 admin admin] PATH_INFO: `/v3/users/84fee63b7c604c88838ee451218afa7f` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:09.254323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47372f03-8c26-4f63-9a8e-9c754f67dae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84fee63b7c604c88838ee451218afa7f)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:09.257679 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47372f03-8c26-4f63-9a8e-9c754f67dae3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:09.536454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-47372f03-8c26-4f63-9a8e-9c754f67dae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84fee63b7c604c88838ee451218afa7f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:09.537306 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-47372f03-8c26-4f63-9a8e-9c754f67dae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84fee63b7c604c88838ee451218afa7f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:09.538086 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-47372f03-8c26-4f63-9a8e-9c754f67dae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84fee63b7c604c88838ee451218afa7f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:09.562893 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-47372f03-8c26-4f63-9a8e-9c754f67dae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84fee63b7c604c88838ee451218afa7f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:09.578515 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-47372f03-8c26-4f63-9a8e-9c754f67dae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84fee63b7c604c88838ee451218afa7f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:09.599404 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b6134-603c-49ec-9b11-4a15b8865ebd admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:09.601898 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8e8b6134-603c-49ec-9b11-4a15b8865ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:09.634682 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b6134-603c-49ec-9b11-4a15b8865ebd admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:09.643529 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d122757-685c-43a0-b912-33683c7a8862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:09.645325 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d122757-685c-43a0-b912-33683c7a8862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:09.645768 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d122757-685c-43a0-b912-33683c7a8862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:09.646166 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d122757-685c-43a0-b912-33683c7a8862 admin admin] PATH_INFO: `/v3/projects/318651762ce54d3b8664622c868e84f1` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:09.648522 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d122757-685c-43a0-b912-33683c7a8862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=318651762ce54d3b8664622c868e84f1)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:09.651049 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d122757-685c-43a0-b912-33683c7a8862 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:09.693564 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 925/1846] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:09 2022] DELETE /identity/v3/users/84fee63b7c604c88838ee451218afa7f => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:09.699520 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a5aa0-593b-4db1-9ea2-ef5eed145350 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:09.700706 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a5aa0-593b-4db1-9ea2-ef5eed145350 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:09.701353 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a5aa0-593b-4db1-9ea2-ef5eed145350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:09.747516 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-6b1a5aa0-593b-4db1-9ea2-ef5eed145350 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:09.750493 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6b1a5aa0-593b-4db1-9ea2-ef5eed145350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:09.832183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8d122757-685c-43a0-b912-33683c7a8862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '318651762ce54d3b8664622c868e84f1', 'request_id': 'req-8d122757-685c-43a0-b912-33683c7a8862'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:09.853135 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8d122757-685c-43a0-b912-33683c7a8862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '318651762ce54d3b8664622c868e84f1', 'request_id': 'req-8d122757-685c-43a0-b912-33683c7a8862'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:09.865538 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8d122757-685c-43a0-b912-33683c7a8862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '318651762ce54d3b8664622c868e84f1', 'request_id': 'req-8d122757-685c-43a0-b912-33683c7a8862'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:09.889690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 926/1847] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:47:09.901096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a5aa0-593b-4db1-9ea2-ef5eed145350 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:09.902835 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6b1a5aa0-593b-4db1-9ea2-ef5eed145350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:09.963605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8d122757-685c-43a0-b912-33683c7a8862 admin admin] The token cache is being invalidate because project 318651762ce54d3b8664622c868e84f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:09.964326 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8d122757-685c-43a0-b912-33683c7a8862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:09.965483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8d122757-685c-43a0-b912-33683c7a8862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:09.972179 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 922/1848] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:09 2022] DELETE /identity/v3/projects/318651762ce54d3b8664622c868e84f1 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:09.979972 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d122757-685c-43a0-b912-33683c7a8862 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:09.982669 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8d122757-685c-43a0-b912-33683c7a8862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:10.004879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a5aa0-593b-4db1-9ea2-ef5eed145350 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:10.079252 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:10.085446 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a71438-c528-4ced-a487-0ca08426b08b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:10.086230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a71438-c528-4ced-a487-0ca08426b08b service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:10.086610 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a71438-c528-4ced-a487-0ca08426b08b service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:10.086961 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a71438-c528-4ced-a487-0ca08426b08b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:10.089194 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-63a71438-c528-4ced-a487-0ca08426b08b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:10.116930 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d122757-685c-43a0-b912-33683c7a8862 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:10.124200 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81527e-b772-4eee-a1c6-81852349ef76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:10.125124 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81527e-b772-4eee-a1c6-81852349ef76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:10.125476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81527e-b772-4eee-a1c6-81852349ef76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:10.126182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81527e-b772-4eee-a1c6-81852349ef76 admin admin] PATH_INFO: `/v3/users/e4851a3c98514217af3cb6a35f3961f9` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:10.127746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa81527e-b772-4eee-a1c6-81852349ef76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4851a3c98514217af3cb6a35f3961f9)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:10.130099 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa81527e-b772-4eee-a1c6-81852349ef76 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:10.135013 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:10.196284 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a71438-c528-4ced-a487-0ca08426b08b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:10.197806 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a71438-c528-4ced-a487-0ca08426b08b service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:10.216478 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 927/1849] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:47:10.493681 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-aa81527e-b772-4eee-a1c6-81852349ef76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4851a3c98514217af3cb6a35f3961f9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:10.495767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-aa81527e-b772-4eee-a1c6-81852349ef76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4851a3c98514217af3cb6a35f3961f9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:10.495957 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-aa81527e-b772-4eee-a1c6-81852349ef76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4851a3c98514217af3cb6a35f3961f9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:10.524685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-aa81527e-b772-4eee-a1c6-81852349ef76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4851a3c98514217af3cb6a35f3961f9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:10.536034 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-aa81527e-b772-4eee-a1c6-81852349ef76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4851a3c98514217af3cb6a35f3961f9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:10.645621 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 923/1850] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:09 2022] DELETE /identity/v3/users/e4851a3c98514217af3cb6a35f3961f9 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:11.043525 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a71438-c528-4ced-a487-0ca08426b08b service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:11.048131 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-63a71438-c528-4ced-a487-0ca08426b08b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:11.100041 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a71438-c528-4ced-a487-0ca08426b08b service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:11.233868 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02831a74-8f4d-45a4-b2bc-0376af4b7494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:11.235786 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02831a74-8f4d-45a4-b2bc-0376af4b7494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:11.236293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02831a74-8f4d-45a4-b2bc-0376af4b7494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:11.236775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02831a74-8f4d-45a4-b2bc-0376af4b7494 admin admin] PATH_INFO: `/v3/projects/54a350b1089242b8864f9d5a50b0c7ba` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:11.239545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02831a74-8f4d-45a4-b2bc-0376af4b7494 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54a350b1089242b8864f9d5a50b0c7ba)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:11.243785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02831a74-8f4d-45a4-b2bc-0376af4b7494 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:11.293673 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81527e-b772-4eee-a1c6-81852349ef76 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:11.296440 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-aa81527e-b772-4eee-a1c6-81852349ef76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:11.319649 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81527e-b772-4eee-a1c6-81852349ef76 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:11.334623 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baedc1a-98c0-4449-947c-ac02a99ce6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:11.335546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baedc1a-98c0-4449-947c-ac02a99ce6af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:11.335970 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baedc1a-98c0-4449-947c-ac02a99ce6af admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:11.336280 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baedc1a-98c0-4449-947c-ac02a99ce6af admin admin] PATH_INFO: `/v3/projects/7827a6f853164504a1a2e5093615953a` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:11.338482 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baedc1a-98c0-4449-947c-ac02a99ce6af admin admin] RBAC: Authorizing `identity:delete_project(project_id=7827a6f853164504a1a2e5093615953a)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:11.341270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baedc1a-98c0-4449-947c-ac02a99ce6af admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:11.518939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-02831a74-8f4d-45a4-b2bc-0376af4b7494 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54a350b1089242b8864f9d5a50b0c7ba', 'request_id': 'req-02831a74-8f4d-45a4-b2bc-0376af4b7494'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:11.526561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:11.565054 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-02831a74-8f4d-45a4-b2bc-0376af4b7494 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54a350b1089242b8864f9d5a50b0c7ba', 'request_id': 'req-02831a74-8f4d-45a4-b2bc-0376af4b7494'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:11.582918 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-02831a74-8f4d-45a4-b2bc-0376af4b7494 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54a350b1089242b8864f9d5a50b0c7ba', 'request_id': 'req-02831a74-8f4d-45a4-b2bc-0376af4b7494'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:11.590350 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6baedc1a-98c0-4449-947c-ac02a99ce6af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7827a6f853164504a1a2e5093615953a', 'request_id': 'req-6baedc1a-98c0-4449-947c-ac02a99ce6af'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:11.601878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6baedc1a-98c0-4449-947c-ac02a99ce6af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7827a6f853164504a1a2e5093615953a', 'request_id': 'req-6baedc1a-98c0-4449-947c-ac02a99ce6af'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:11.612983 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6baedc1a-98c0-4449-947c-ac02a99ce6af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7827a6f853164504a1a2e5093615953a', 'request_id': 'req-6baedc1a-98c0-4449-947c-ac02a99ce6af'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:11.700670 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-02831a74-8f4d-45a4-b2bc-0376af4b7494 admin admin] The token cache is being invalidate because project 54a350b1089242b8864f9d5a50b0c7ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:11.701412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-02831a74-8f4d-45a4-b2bc-0376af4b7494 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:11.702349 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-02831a74-8f4d-45a4-b2bc-0376af4b7494 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:11.709969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 928/1851] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:11 2022] DELETE /identity/v3/projects/54a350b1089242b8864f9d5a50b0c7ba => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:11.716370 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02831a74-8f4d-45a4-b2bc-0376af4b7494 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:11.717964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-02831a74-8f4d-45a4-b2bc-0376af4b7494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:11.803506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6baedc1a-98c0-4449-947c-ac02a99ce6af admin admin] The token cache is being invalidate because project 7827a6f853164504a1a2e5093615953a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:11.804358 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6baedc1a-98c0-4449-947c-ac02a99ce6af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:11.811565 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6baedc1a-98c0-4449-947c-ac02a99ce6af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:11.825696 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 924/1852] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:11 2022] DELETE /identity/v3/projects/7827a6f853164504a1a2e5093615953a => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:11.828918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfd710-9c40-410d-8d3b-970aaaa72265 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:11.829827 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfd710-9c40-410d-8d3b-970aaaa72265 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:11.830609 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfd710-9c40-410d-8d3b-970aaaa72265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:11.847577 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02831a74-8f4d-45a4-b2bc-0376af4b7494 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:11.908240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d2bfd710-9c40-410d-8d3b-970aaaa72265 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:11.911359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d2bfd710-9c40-410d-8d3b-970aaaa72265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:11.992912 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b243a6-ae70-4ad4-acc6-7755477c95fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:11.996062 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b243a6-ae70-4ad4-acc6-7755477c95fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:11.996553 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b243a6-ae70-4ad4-acc6-7755477c95fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:11.996854 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b243a6-ae70-4ad4-acc6-7755477c95fe admin admin] PATH_INFO: `/v3/users/c7e0e39a501640828863644d19a55e02` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:12.004819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b243a6-ae70-4ad4-acc6-7755477c95fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7e0e39a501640828863644d19a55e02)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:12.011499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b243a6-ae70-4ad4-acc6-7755477c95fe admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:12.197671 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 925/1853] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:12.207864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d9c2b-287d-4abe-84fa-f35259b5c949 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:12.208386 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d9c2b-287d-4abe-84fa-f35259b5c949 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:12.208839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d9c2b-287d-4abe-84fa-f35259b5c949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:12.235882 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:12.274749 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-626d9c2b-287d-4abe-84fa-f35259b5c949 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:12.276766 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-626d9c2b-287d-4abe-84fa-f35259b5c949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:12.300792 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 926/1854] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:12.302360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d9c2b-287d-4abe-84fa-f35259b5c949 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:12.320017 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d9c2b-287d-4abe-84fa-f35259b5c949 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:12.330562 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fc221-7e88-4455-a5c2-b742b71c974b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:12.332932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fc221-7e88-4455-a5c2-b742b71c974b admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:12.333367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fc221-7e88-4455-a5c2-b742b71c974b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:12.335735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fc221-7e88-4455-a5c2-b742b71c974b admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:12.339259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64fc221-7e88-4455-a5c2-b742b71c974b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:47:12.339803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64fc221-7e88-4455-a5c2-b742b71c974b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:12.347244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64fc221-7e88-4455-a5c2-b742b71c974b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:12.385383 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-64b243a6-ae70-4ad4-acc6-7755477c95fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7e0e39a501640828863644d19a55e02'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:12.388285 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 927/1855] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:47:12.389059 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-64b243a6-ae70-4ad4-acc6-7755477c95fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7e0e39a501640828863644d19a55e02'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:12.389283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fc221-7e88-4455-a5c2-b742b71c974b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:12.390645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-64b243a6-ae70-4ad4-acc6-7755477c95fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7e0e39a501640828863644d19a55e02'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:12.424064 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fc221-7e88-4455-a5c2-b742b71c974b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:12.442463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defac774-35f5-4480-8185-8a8432f32731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:12.444639 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defac774-35f5-4480-8185-8a8432f32731 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:12.446584 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defac774-35f5-4480-8185-8a8432f32731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:12.447532 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defac774-35f5-4480-8185-8a8432f32731 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:12.449833 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defac774-35f5-4480-8185-8a8432f32731 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:12.456942 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defac774-35f5-4480-8185-8a8432f32731 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:12.486329 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-64b243a6-ae70-4ad4-acc6-7755477c95fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7e0e39a501640828863644d19a55e02'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:12.522537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-64b243a6-ae70-4ad4-acc6-7755477c95fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7e0e39a501640828863644d19a55e02'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:12.680662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 929/1856] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:11 2022] DELETE /identity/v3/users/c7e0e39a501640828863644d19a55e02 => generated 0 bytes in 965 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:12.693770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b243a6-ae70-4ad4-acc6-7755477c95fe admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:12.694495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 928/1857] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:47:12 2022] POST /identity/v3/projects => generated 383 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:12.696414 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-64b243a6-ae70-4ad4-acc6-7755477c95fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:12.704619 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:12.774187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b243a6-ae70-4ad4-acc6-7755477c95fe admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:12.878943 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50e213-975b-41c1-b3d1-0c178957969e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:12.880863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50e213-975b-41c1-b3d1-0c178957969e admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:12.881302 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50e213-975b-41c1-b3d1-0c178957969e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:12.881712 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50e213-975b-41c1-b3d1-0c178957969e admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:12.884527 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50e213-975b-41c1-b3d1-0c178957969e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:12.891693 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50e213-975b-41c1-b3d1-0c178957969e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:13.164577 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 930/1858] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:47:12 2022] POST /identity/v3/users => generated 292 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:13.190548 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defac774-35f5-4480-8185-8a8432f32731 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:13.207462 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defac774-35f5-4480-8185-8a8432f32731 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:13.213620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea460b-a62b-4e98-a30d-16bb534a596c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:13.214332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea460b-a62b-4e98-a30d-16bb534a596c admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:13.214612 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea460b-a62b-4e98-a30d-16bb534a596c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:13.214887 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea460b-a62b-4e98-a30d-16bb534a596c admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:13.215804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea460b-a62b-4e98-a30d-16bb534a596c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:13.217661 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea460b-a62b-4e98-a30d-16bb534a596c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:13.235488 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 929/1859] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:13.257729 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50e213-975b-41c1-b3d1-0c178957969e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:13.275261 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50e213-975b-41c1-b3d1-0c178957969e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:13.281464 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6b16e-0f19-4430-bab5-e204fc33d9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:13.282259 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6b16e-0f19-4430-bab5-e204fc33d9b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:13.282601 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6b16e-0f19-4430-bab5-e204fc33d9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:13.282942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6b16e-0f19-4430-bab5-e204fc33d9b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:13.284167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6b16e-0f19-4430-bab5-e204fc33d9b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:13.285985 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6b16e-0f19-4430-bab5-e204fc33d9b5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:13.301441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 931/1860] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:13.309196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea460b-a62b-4e98-a30d-16bb534a596c admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:13.323359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea460b-a62b-4e98-a30d-16bb534a596c admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:13.332663 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bb463-8b17-4ef1-8f64-a04b6ef8fb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:13.334381 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bb463-8b17-4ef1-8f64-a04b6ef8fb59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:13.335827 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bb463-8b17-4ef1-8f64-a04b6ef8fb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:13.336689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bb463-8b17-4ef1-8f64-a04b6ef8fb59 admin admin] PATH_INFO: `/v3/projects/a8e1978e517b49678fbe4d79583eac10/users/523248ec76614252a9811b4f1595e1b9/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:13.362808 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bb463-8b17-4ef1-8f64-a04b6ef8fb59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8e1978e517b49678fbe4d79583eac10, user_id=523248ec76614252a9811b4f1595e1b9, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:13.365922 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bb463-8b17-4ef1-8f64-a04b6ef8fb59 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:13.621687 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 930/1861] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:13 2022] PUT /identity/v3/projects/a8e1978e517b49678fbe4d79583eac10/users/523248ec76614252a9811b4f1595e1b9/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:13.636320 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c095dd-45a4-4acd-b076-3522355fa1cb None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:13.637152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c095dd-45a4-4acd-b076-3522355fa1cb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:13.637152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c095dd-45a4-4acd-b076-3522355fa1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:13.680130 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-89c095dd-45a4-4acd-b076-3522355fa1cb None None] MFA Rules not processed for user `523248ec76614252a9811b4f1595e1b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:13.682191 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-89c095dd-45a4-4acd-b076-3522355fa1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:13.733908 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bb463-8b17-4ef1-8f64-a04b6ef8fb59 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:13.735932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-aa3bb463-8b17-4ef1-8f64-a04b6ef8fb59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:13.810982 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 932/1862] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:13.817597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c095dd-45a4-4acd-b076-3522355fa1cb None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:13.864320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bb463-8b17-4ef1-8f64-a04b6ef8fb59 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:13.873639 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f25afe-cacf-465f-b493-e939e760aa1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:13.874667 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f25afe-cacf-465f-b493-e939e760aa1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:13.875054 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f25afe-cacf-465f-b493-e939e760aa1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:13.881126 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f25afe-cacf-465f-b493-e939e760aa1b admin admin] PATH_INFO: `/v3/projects/edd96bfbf463468d90645f74d2397c5c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:13.882363 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f25afe-cacf-465f-b493-e939e760aa1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=edd96bfbf463468d90645f74d2397c5c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:13.884498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f25afe-cacf-465f-b493-e939e760aa1b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:13.931693 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c095dd-45a4-4acd-b076-3522355fa1cb None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:13.937850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26de1f-b9dc-4b2f-89ed-0400c45aa45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:13.938685 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26de1f-b9dc-4b2f-89ed-0400c45aa45b admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:13.938981 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26de1f-b9dc-4b2f-89ed-0400c45aa45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:13.939288 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26de1f-b9dc-4b2f-89ed-0400c45aa45b admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:13.940018 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26de1f-b9dc-4b2f-89ed-0400c45aa45b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:13.941993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26de1f-b9dc-4b2f-89ed-0400c45aa45b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:14.030708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-25f25afe-cacf-465f-b493-e939e760aa1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edd96bfbf463468d90645f74d2397c5c', 'request_id': 'req-25f25afe-cacf-465f-b493-e939e760aa1b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:14.037135 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-25f25afe-cacf-465f-b493-e939e760aa1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edd96bfbf463468d90645f74d2397c5c', 'request_id': 'req-25f25afe-cacf-465f-b493-e939e760aa1b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:14.045575 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-25f25afe-cacf-465f-b493-e939e760aa1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edd96bfbf463468d90645f74d2397c5c', 'request_id': 'req-25f25afe-cacf-465f-b493-e939e760aa1b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:14.096215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 933/1863] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:47:13 2022] POST /identity/v3/projects => generated 383 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:14.097575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26de1f-b9dc-4b2f-89ed-0400c45aa45b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:14.208671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26de1f-b9dc-4b2f-89ed-0400c45aa45b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:14.213887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2da4a-8f40-4250-af50-9c2beeb6f63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:14.214675 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2da4a-8f40-4250-af50-9c2beeb6f63c admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:14.214996 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2da4a-8f40-4250-af50-9c2beeb6f63c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:14.215319 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2da4a-8f40-4250-af50-9c2beeb6f63c admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:14.217819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2da4a-8f40-4250-af50-9c2beeb6f63c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:14.218996 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2da4a-8f40-4250-af50-9c2beeb6f63c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:14.240406 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-25f25afe-cacf-465f-b493-e939e760aa1b admin admin] The token cache is being invalidate because project edd96bfbf463468d90645f74d2397c5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:14.240406 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-25f25afe-cacf-465f-b493-e939e760aa1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:14.240406 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-25f25afe-cacf-465f-b493-e939e760aa1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:14.241750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 931/1864] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:13 2022] DELETE /identity/v3/projects/edd96bfbf463468d90645f74d2397c5c => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:14.248032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1acd1f-c92d-4178-a641-98e94cda2986 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:14.248418 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1acd1f-c92d-4178-a641-98e94cda2986 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:14.248707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1acd1f-c92d-4178-a641-98e94cda2986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:14.271162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-9b1acd1f-c92d-4178-a641-98e94cda2986 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:14.272528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9b1acd1f-c92d-4178-a641-98e94cda2986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:14.283036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 932/1865] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:47:14.291217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cde707-32f4-411d-91cb-503c32013bfe None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:14.291535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cde707-32f4-411d-91cb-503c32013bfe None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:14.291969 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cde707-32f4-411d-91cb-503c32013bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:14.312938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-59cde707-32f4-411d-91cb-503c32013bfe None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:14.314378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-59cde707-32f4-411d-91cb-503c32013bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:14.324781 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 933/1866] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:14.331008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cde707-32f4-411d-91cb-503c32013bfe None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:14.347891 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:14.347891 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cde707-32f4-411d-91cb-503c32013bfe None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:14.351599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71dae11-047f-40ba-919f-fa06f45fa738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:14.352311 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71dae11-047f-40ba-919f-fa06f45fa738 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:14.352593 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71dae11-047f-40ba-919f-fa06f45fa738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:14.352868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71dae11-047f-40ba-919f-fa06f45fa738 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:14.353610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71dae11-047f-40ba-919f-fa06f45fa738 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:47:14.353915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71dae11-047f-40ba-919f-fa06f45fa738 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:14.355529 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71dae11-047f-40ba-919f-fa06f45fa738 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:14.370635 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 934/1867] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:47:14 2022] POST /identity/v3/users => generated 298 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:14.371783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 934/1868] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:47:14.376613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71dae11-047f-40ba-919f-fa06f45fa738 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:14.376801 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2da4a-8f40-4250-af50-9c2beeb6f63c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:14.378514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b71dae11-047f-40ba-919f-fa06f45fa738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:14.387770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2da4a-8f40-4250-af50-9c2beeb6f63c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:14.393222 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eca7b6-d4ef-4b0c-be40-50db4df04326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:14.393906 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eca7b6-d4ef-4b0c-be40-50db4df04326 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:14.394188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eca7b6-d4ef-4b0c-be40-50db4df04326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:14.394461 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eca7b6-d4ef-4b0c-be40-50db4df04326 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:14.395223 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eca7b6-d4ef-4b0c-be40-50db4df04326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:14.395449 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71dae11-047f-40ba-919f-fa06f45fa738 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:14.397349 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eca7b6-d4ef-4b0c-be40-50db4df04326 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:14.401659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270c3b1-8553-4b70-8ea2-dec7d3a3dacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:14.402614 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270c3b1-8553-4b70-8ea2-dec7d3a3dacc admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:14.403054 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270c3b1-8553-4b70-8ea2-dec7d3a3dacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:14.404730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270c3b1-8553-4b70-8ea2-dec7d3a3dacc admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:14.405249 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270c3b1-8553-4b70-8ea2-dec7d3a3dacc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:14.407368 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270c3b1-8553-4b70-8ea2-dec7d3a3dacc admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:14.423190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 935/1869] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:14.427913 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270c3b1-8553-4b70-8ea2-dec7d3a3dacc admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:14.439256 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270c3b1-8553-4b70-8ea2-dec7d3a3dacc admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:14.444267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82424ab-7dad-4657-8a68-335967309acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:14.445187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82424ab-7dad-4657-8a68-335967309acc admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:14.445463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82424ab-7dad-4657-8a68-335967309acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:14.447878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82424ab-7dad-4657-8a68-335967309acc admin admin] PATH_INFO: `/v3/projects/5eaaccb93ff34bad9d6f4fa4bc3f24e4/users/5dfc5fe8cd5e4f388b50b1fb0ca6687a/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:14.466441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82424ab-7dad-4657-8a68-335967309acc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eaaccb93ff34bad9d6f4fa4bc3f24e4, user_id=5dfc5fe8cd5e4f388b50b1fb0ca6687a, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:14.470333 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82424ab-7dad-4657-8a68-335967309acc admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:14.509619 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 935/1870] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:47:14 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:14.514357 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eca7b6-d4ef-4b0c-be40-50db4df04326 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:14.569039 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 936/1871] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:14 2022] PUT /identity/v3/projects/5eaaccb93ff34bad9d6f4fa4bc3f24e4/users/5dfc5fe8cd5e4f388b50b1fb0ca6687a/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:14.573095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82424ab-7dad-4657-8a68-335967309acc admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:14.600620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eca7b6-d4ef-4b0c-be40-50db4df04326 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:14.692141 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82424ab-7dad-4657-8a68-335967309acc admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:14.697816 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfe33c-2a33-4350-96da-4580b1380e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:14.698678 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d156fe-2c07-4605-8ad9-36a2b43ed04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:14.698822 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfe33c-2a33-4350-96da-4580b1380e3d admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:14.699077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfe33c-2a33-4350-96da-4580b1380e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:14.699357 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfe33c-2a33-4350-96da-4580b1380e3d admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:14.699493 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d156fe-2c07-4605-8ad9-36a2b43ed04d admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:14.699910 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d156fe-2c07-4605-8ad9-36a2b43ed04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:14.700707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfe33c-2a33-4350-96da-4580b1380e3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:14.700883 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d156fe-2c07-4605-8ad9-36a2b43ed04d admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:14.702165 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d156fe-2c07-4605-8ad9-36a2b43ed04d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:14.702966 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfe33c-2a33-4350-96da-4580b1380e3d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:14.704131 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d156fe-2c07-4605-8ad9-36a2b43ed04d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:14.718653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 937/1872] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:47:14.720881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d156fe-2c07-4605-8ad9-36a2b43ed04d admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:14.731366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d156fe-2c07-4605-8ad9-36a2b43ed04d admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:14.736915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f65ba3-a2eb-4792-9a38-b944b143347a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:14.737836 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f65ba3-a2eb-4792-9a38-b944b143347a admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:14.738199 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f65ba3-a2eb-4792-9a38-b944b143347a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:14.743953 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f65ba3-a2eb-4792-9a38-b944b143347a admin admin] PATH_INFO: `/v3/projects/5eaaccb93ff34bad9d6f4fa4bc3f24e4/users/5dfc5fe8cd5e4f388b50b1fb0ca6687a/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:14.743953 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f65ba3-a2eb-4792-9a38-b944b143347a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eaaccb93ff34bad9d6f4fa4bc3f24e4, user_id=5dfc5fe8cd5e4f388b50b1fb0ca6687a, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:14.743953 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f65ba3-a2eb-4792-9a38-b944b143347a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:14.755304 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 938/1873] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:14 2022] PUT /identity/v3/projects/5eaaccb93ff34bad9d6f4fa4bc3f24e4/users/5dfc5fe8cd5e4f388b50b1fb0ca6687a/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:14.781910 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652be5a-08d5-4308-a081-ba5ac19e31a4 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:14.782367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652be5a-08d5-4308-a081-ba5ac19e31a4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:14.782828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652be5a-08d5-4308-a081-ba5ac19e31a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:14.832715 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-3652be5a-08d5-4308-a081-ba5ac19e31a4 None None] MFA Rules not processed for user `5dfc5fe8cd5e4f388b50b1fb0ca6687a`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:14.834716 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3652be5a-08d5-4308-a081-ba5ac19e31a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:14.867874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 936/1874] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:47:14 2022] POST /identity/v3/users => generated 290 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:47:14.878219 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfe33c-2a33-4350-96da-4580b1380e3d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:14.900268 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfe33c-2a33-4350-96da-4580b1380e3d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:14.942694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 939/1875] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:47:14.955878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652be5a-08d5-4308-a081-ba5ac19e31a4 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:14.958048 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3652be5a-08d5-4308-a081-ba5ac19e31a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:15.011717 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec62ed-3980-4c1a-8c2a-e7458f3f9f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:15.013859 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec62ed-3980-4c1a-8c2a-e7458f3f9f74 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:15.014374 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec62ed-3980-4c1a-8c2a-e7458f3f9f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:15.014815 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec62ed-3980-4c1a-8c2a-e7458f3f9f74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:15.015453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec62ed-3980-4c1a-8c2a-e7458f3f9f74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:15.018063 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec62ed-3980-4c1a-8c2a-e7458f3f9f74 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:15.035295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 937/1876] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:15.039785 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652be5a-08d5-4308-a081-ba5ac19e31a4 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:15.045225 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f75639-9678-43f1-b6f5-a96bdf38fe9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:15.046032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f75639-9678-43f1-b6f5-a96bdf38fe9f service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:15.046387 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f75639-9678-43f1-b6f5-a96bdf38fe9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:15.046828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f75639-9678-43f1-b6f5-a96bdf38fe9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:15.048850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec62ed-3980-4c1a-8c2a-e7458f3f9f74 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:15.049285 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-44f75639-9678-43f1-b6f5-a96bdf38fe9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:15.072137 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec62ed-3980-4c1a-8c2a-e7458f3f9f74 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:15.078139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a01c65-cf09-47dc-b25f-5e4c389769d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:15.079130 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a01c65-cf09-47dc-b25f-5e4c389769d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:15.079273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a01c65-cf09-47dc-b25f-5e4c389769d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:15.079508 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a01c65-cf09-47dc-b25f-5e4c389769d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:15.080460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a01c65-cf09-47dc-b25f-5e4c389769d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:15.082410 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a01c65-cf09-47dc-b25f-5e4c389769d1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:15.097553 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 938/1877] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:15.112590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a01c65-cf09-47dc-b25f-5e4c389769d1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:15.131308 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a01c65-cf09-47dc-b25f-5e4c389769d1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:15.136905 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75ec7d-dfe3-47a3-817b-9cb76c0705af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:15.138068 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75ec7d-dfe3-47a3-817b-9cb76c0705af admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:15.138650 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75ec7d-dfe3-47a3-817b-9cb76c0705af admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:15.139104 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75ec7d-dfe3-47a3-817b-9cb76c0705af admin admin] PATH_INFO: `/v3/projects/9fc440827cce41c892138429c791e59d/users/dfe1e7d5670d4aa09d8cdf5e038711b3/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:15.170603 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75ec7d-dfe3-47a3-817b-9cb76c0705af admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fc440827cce41c892138429c791e59d, user_id=dfe1e7d5670d4aa09d8cdf5e038711b3, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:15.177277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f75639-9678-43f1-b6f5-a96bdf38fe9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:15.178034 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75ec7d-dfe3-47a3-817b-9cb76c0705af admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:15.178759 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f75639-9678-43f1-b6f5-a96bdf38fe9f service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:15.189326 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 940/1878] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:47:15.210285 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:15.264608 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 939/1879] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:15 2022] PUT /identity/v3/projects/9fc440827cce41c892138429c791e59d/users/dfe1e7d5670d4aa09d8cdf5e038711b3/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:15.286559 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd270c-efbe-4ac2-9b57-c5589b651816 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:15.287413 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd270c-efbe-4ac2-9b57-c5589b651816 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:15.287413 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd270c-efbe-4ac2-9b57-c5589b651816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:15.330234 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-46cd270c-efbe-4ac2-9b57-c5589b651816 None None] MFA Rules not processed for user `dfe1e7d5670d4aa09d8cdf5e038711b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:15.331723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-46cd270c-efbe-4ac2-9b57-c5589b651816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:15.417169 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 941/1880] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:15.422628 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75ec7d-dfe3-47a3-817b-9cb76c0705af admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:15.455444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:15.517556 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:15.554749 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75ec7d-dfe3-47a3-817b-9cb76c0705af admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:15.560212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d1714-73dc-4374-ad91-2005a1f74c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:15.561010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d1714-73dc-4374-ad91-2005a1f74c16 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:15.573187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d1714-73dc-4374-ad91-2005a1f74c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:15.573187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d1714-73dc-4374-ad91-2005a1f74c16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:15.573187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d1714-73dc-4374-ad91-2005a1f74c16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:15.573187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d1714-73dc-4374-ad91-2005a1f74c16 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:15.687794 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 940/1881] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:47:15 2022] POST /identity/v3/projects => generated 381 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:47:15.689311 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd270c-efbe-4ac2-9b57-c5589b651816 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:15.701434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd270c-efbe-4ac2-9b57-c5589b651816 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:15.771318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f63eb-e6a9-41bb-9a82-37bb99c13dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:15.772060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f63eb-e6a9-41bb-9a82-37bb99c13dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:15.772347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f63eb-e6a9-41bb-9a82-37bb99c13dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:15.772684 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f63eb-e6a9-41bb-9a82-37bb99c13dd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:15.773402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f63eb-e6a9-41bb-9a82-37bb99c13dd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:15.775440 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f63eb-e6a9-41bb-9a82-37bb99c13dd6 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:15.955426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 942/1882] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:47:15 2022] POST /identity/v3/users => generated 297 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:47:15.956891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d1714-73dc-4374-ad91-2005a1f74c16 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:15.972588 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d1714-73dc-4374-ad91-2005a1f74c16 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:15.984147 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ad05d-d70b-4260-8b87-e1bff2d3f4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:15.984147 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ad05d-d70b-4260-8b87-e1bff2d3f4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:15.984147 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ad05d-d70b-4260-8b87-e1bff2d3f4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:15.984566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ad05d-d70b-4260-8b87-e1bff2d3f4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:15.985905 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ad05d-d70b-4260-8b87-e1bff2d3f4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:15.988963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ad05d-d70b-4260-8b87-e1bff2d3f4d6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:16.011648 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 941/1883] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:16.021617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f63eb-e6a9-41bb-9a82-37bb99c13dd6 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:16.033017 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f63eb-e6a9-41bb-9a82-37bb99c13dd6 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:16.038351 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66c8df-b112-4602-8f3a-422d33fd1d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:16.039346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66c8df-b112-4602-8f3a-422d33fd1d5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:16.039638 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66c8df-b112-4602-8f3a-422d33fd1d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:16.039983 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66c8df-b112-4602-8f3a-422d33fd1d5a admin admin] PATH_INFO: `/v3/projects/464e96fe2f4740a2a187304d1b7b2bc5/users/24075120ac3f42268949ffc39dc47f75/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:16.045911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:16.060302 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66c8df-b112-4602-8f3a-422d33fd1d5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=464e96fe2f4740a2a187304d1b7b2bc5, user_id=24075120ac3f42268949ffc39dc47f75, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:16.063031 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66c8df-b112-4602-8f3a-422d33fd1d5a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:16.118273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:16.192943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 943/1884] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:16 2022] PUT /identity/v3/projects/464e96fe2f4740a2a187304d1b7b2bc5/users/24075120ac3f42268949ffc39dc47f75/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:16.197024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ad05d-d70b-4260-8b87-e1bff2d3f4d6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:16.210078 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ad05d-d70b-4260-8b87-e1bff2d3f4d6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:16.324565 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fddc02-6c71-435f-85b9-a011398c1066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:16.326662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fddc02-6c71-435f-85b9-a011398c1066 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:16.327049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fddc02-6c71-435f-85b9-a011398c1066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:16.328110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fddc02-6c71-435f-85b9-a011398c1066 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:16.329788 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fddc02-6c71-435f-85b9-a011398c1066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:16.334172 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fddc02-6c71-435f-85b9-a011398c1066 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:16.356993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 942/1885] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:47:16.359209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66c8df-b112-4602-8f3a-422d33fd1d5a admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:16.372330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66c8df-b112-4602-8f3a-422d33fd1d5a admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:16.376768 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9f15f-4247-4e67-9198-7a65383060dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:16.377724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9f15f-4247-4e67-9198-7a65383060dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:16.378034 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9f15f-4247-4e67-9198-7a65383060dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:16.378322 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9f15f-4247-4e67-9198-7a65383060dc admin admin] PATH_INFO: `/v3/projects/464e96fe2f4740a2a187304d1b7b2bc5/users/24075120ac3f42268949ffc39dc47f75/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:16.380489 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc9f15f-4247-4e67-9198-7a65383060dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=464e96fe2f4740a2a187304d1b7b2bc5, user_id=24075120ac3f42268949ffc39dc47f75, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:16.383198 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc9f15f-4247-4e67-9198-7a65383060dc admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:16.395554 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 944/1886] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:16 2022] PUT /identity/v3/projects/464e96fe2f4740a2a187304d1b7b2bc5/users/24075120ac3f42268949ffc39dc47f75/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:16.408136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bcb35-a412-4261-863b-49f15b52fdf2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:16.408677 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bcb35-a412-4261-863b-49f15b52fdf2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:16.408965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bcb35-a412-4261-863b-49f15b52fdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:16.441934 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-091bcb35-a412-4261-863b-49f15b52fdf2 None None] MFA Rules not processed for user `24075120ac3f42268949ffc39dc47f75`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:16.443366 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-091bcb35-a412-4261-863b-49f15b52fdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:16.543519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 943/1887] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:16.555606 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9f15f-4247-4e67-9198-7a65383060dc admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:16.658893 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9f15f-4247-4e67-9198-7a65383060dc admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:16.663112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7033e8-7148-41bc-9109-980096bb4161 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:16.663823 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7033e8-7148-41bc-9109-980096bb4161 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:16.664159 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7033e8-7148-41bc-9109-980096bb4161 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:16.664754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7033e8-7148-41bc-9109-980096bb4161 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:16.677230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7033e8-7148-41bc-9109-980096bb4161 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:16.678698 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7033e8-7148-41bc-9109-980096bb4161 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:16.690812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 945/1888] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:47:17.432441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bcb35-a412-4261-863b-49f15b52fdf2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:17.551533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bcb35-a412-4261-863b-49f15b52fdf2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:17.561309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b197a65-9152-4c51-be31-9b666ae5ba98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:17.562335 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b197a65-9152-4c51-be31-9b666ae5ba98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:17.562936 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b197a65-9152-4c51-be31-9b666ae5ba98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:17.563422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b197a65-9152-4c51-be31-9b666ae5ba98 admin admin] PATH_INFO: `/v3/users/dfe1e7d5670d4aa09d8cdf5e038711b3` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:17.566184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b197a65-9152-4c51-be31-9b666ae5ba98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfe1e7d5670d4aa09d8cdf5e038711b3)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:17.571494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b197a65-9152-4c51-be31-9b666ae5ba98 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:17.586719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:17.793119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2b197a65-9152-4c51-be31-9b666ae5ba98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfe1e7d5670d4aa09d8cdf5e038711b3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:17.793896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2b197a65-9152-4c51-be31-9b666ae5ba98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfe1e7d5670d4aa09d8cdf5e038711b3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:17.794743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2b197a65-9152-4c51-be31-9b666ae5ba98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfe1e7d5670d4aa09d8cdf5e038711b3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:17.813157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2b197a65-9152-4c51-be31-9b666ae5ba98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfe1e7d5670d4aa09d8cdf5e038711b3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:17.852020 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2b197a65-9152-4c51-be31-9b666ae5ba98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfe1e7d5670d4aa09d8cdf5e038711b3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:17.969492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 944/1889] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:17 2022] DELETE /identity/v3/users/dfe1e7d5670d4aa09d8cdf5e038711b3 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:17.974291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d3b7a-ea5e-4e6f-9e1c-eaedb559343f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:17.974861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d3b7a-ea5e-4e6f-9e1c-eaedb559343f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:17.974861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d3b7a-ea5e-4e6f-9e1c-eaedb559343f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:18.000840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2c0d3b7a-ea5e-4e6f-9e1c-eaedb559343f None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:18.003131 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2c0d3b7a-ea5e-4e6f-9e1c-eaedb559343f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:18.135157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 946/1890] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:18.154695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b197a65-9152-4c51-be31-9b666ae5ba98 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:18.156541 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2b197a65-9152-4c51-be31-9b666ae5ba98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:18.301609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b197a65-9152-4c51-be31-9b666ae5ba98 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:18.311159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97e624-1c3a-4e8d-ae08-5c09496fe263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:18.311159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97e624-1c3a-4e8d-ae08-5c09496fe263 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:18.311159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97e624-1c3a-4e8d-ae08-5c09496fe263 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:18.311159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97e624-1c3a-4e8d-ae08-5c09496fe263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:18.320226 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb97e624-1c3a-4e8d-ae08-5c09496fe263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:18.321759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb97e624-1c3a-4e8d-ae08-5c09496fe263 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:18.336034 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 945/1891] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:47:19.029313 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d3b7a-ea5e-4e6f-9e1c-eaedb559343f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:19.030873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2c0d3b7a-ea5e-4e6f-9e1c-eaedb559343f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:19.044617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d3b7a-ea5e-4e6f-9e1c-eaedb559343f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:19.052178 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6bb016-7ace-432e-b9f7-c92262698e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:19.053106 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bb016-7ace-432e-b9f7-c92262698e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:19.053561 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bb016-7ace-432e-b9f7-c92262698e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:19.053990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bb016-7ace-432e-b9f7-c92262698e53 admin admin] PATH_INFO: `/v3/projects/9fc440827cce41c892138429c791e59d` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:19.055555 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6bb016-7ace-432e-b9f7-c92262698e53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fc440827cce41c892138429c791e59d)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:19.065558 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6bb016-7ace-432e-b9f7-c92262698e53 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:19.268721 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-cb6bb016-7ace-432e-b9f7-c92262698e53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fc440827cce41c892138429c791e59d', 'request_id': 'req-cb6bb016-7ace-432e-b9f7-c92262698e53'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:19.281034 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-cb6bb016-7ace-432e-b9f7-c92262698e53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fc440827cce41c892138429c791e59d', 'request_id': 'req-cb6bb016-7ace-432e-b9f7-c92262698e53'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:19.282801 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:19.290704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-cb6bb016-7ace-432e-b9f7-c92262698e53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fc440827cce41c892138429c791e59d', 'request_id': 'req-cb6bb016-7ace-432e-b9f7-c92262698e53'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:19.394170 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-cb6bb016-7ace-432e-b9f7-c92262698e53 admin admin] The token cache is being invalidate because project 9fc440827cce41c892138429c791e59d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:19.395704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-cb6bb016-7ace-432e-b9f7-c92262698e53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:19.397206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-cb6bb016-7ace-432e-b9f7-c92262698e53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:19.400895 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 947/1892] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:19 2022] DELETE /identity/v3/projects/9fc440827cce41c892138429c791e59d => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:19.407732 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97e624-1c3a-4e8d-ae08-5c09496fe263 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:19.434607 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97e624-1c3a-4e8d-ae08-5c09496fe263 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:19.570369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08012025-304f-4b93-9ee3-c251f8258116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:19.571736 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08012025-304f-4b93-9ee3-c251f8258116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:19.572216 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08012025-304f-4b93-9ee3-c251f8258116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:19.572690 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08012025-304f-4b93-9ee3-c251f8258116 admin admin] PATH_INFO: `/v3/users/24075120ac3f42268949ffc39dc47f75` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:19.574885 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08012025-304f-4b93-9ee3-c251f8258116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24075120ac3f42268949ffc39dc47f75)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:19.580164 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08012025-304f-4b93-9ee3-c251f8258116 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:19.840411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:19.856058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-08012025-304f-4b93-9ee3-c251f8258116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24075120ac3f42268949ffc39dc47f75'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:19.858843 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-08012025-304f-4b93-9ee3-c251f8258116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24075120ac3f42268949ffc39dc47f75'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:19.858843 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-08012025-304f-4b93-9ee3-c251f8258116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24075120ac3f42268949ffc39dc47f75'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:19.875010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-08012025-304f-4b93-9ee3-c251f8258116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24075120ac3f42268949ffc39dc47f75'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:19.883511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-08012025-304f-4b93-9ee3-c251f8258116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24075120ac3f42268949ffc39dc47f75'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:19.993469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 946/1893] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:19 2022] DELETE /identity/v3/users/24075120ac3f42268949ffc39dc47f75 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:21.363140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6bb016-7ace-432e-b9f7-c92262698e53 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:21.366694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-cb6bb016-7ace-432e-b9f7-c92262698e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:21.416130 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6bb016-7ace-432e-b9f7-c92262698e53 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:21.582194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fac60-354d-45e0-ae13-859e1b608a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:21.582194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fac60-354d-45e0-ae13-859e1b608a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:21.582194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fac60-354d-45e0-ae13-859e1b608a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:21.582194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fac60-354d-45e0-ae13-859e1b608a2d admin admin] PATH_INFO: `/v3/projects/464e96fe2f4740a2a187304d1b7b2bc5` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:21.583535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316fac60-354d-45e0-ae13-859e1b608a2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=464e96fe2f4740a2a187304d1b7b2bc5)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:21.586143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316fac60-354d-45e0-ae13-859e1b608a2d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:21.692637 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08012025-304f-4b93-9ee3-c251f8258116 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:21.694386 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-08012025-304f-4b93-9ee3-c251f8258116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:21.834560 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-316fac60-354d-45e0-ae13-859e1b608a2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '464e96fe2f4740a2a187304d1b7b2bc5', 'request_id': 'req-316fac60-354d-45e0-ae13-859e1b608a2d'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:21.844671 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-316fac60-354d-45e0-ae13-859e1b608a2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '464e96fe2f4740a2a187304d1b7b2bc5', 'request_id': 'req-316fac60-354d-45e0-ae13-859e1b608a2d'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:21.869387 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-316fac60-354d-45e0-ae13-859e1b608a2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '464e96fe2f4740a2a187304d1b7b2bc5', 'request_id': 'req-316fac60-354d-45e0-ae13-859e1b608a2d'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:21.901249 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08012025-304f-4b93-9ee3-c251f8258116 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:21.976227 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-316fac60-354d-45e0-ae13-859e1b608a2d admin admin] The token cache is being invalidate because project 464e96fe2f4740a2a187304d1b7b2bc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:21.976542 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-316fac60-354d-45e0-ae13-859e1b608a2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:21.977778 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-316fac60-354d-45e0-ae13-859e1b608a2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:21.979651 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 948/1894] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:21 2022] DELETE /identity/v3/projects/464e96fe2f4740a2a187304d1b7b2bc5 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:21.986205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8687f5e-1c79-4ab6-aea4-34806c95946f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:21.986799 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8687f5e-1c79-4ab6-aea4-34806c95946f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:21.988279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8687f5e-1c79-4ab6-aea4-34806c95946f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:22.054462 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-e8687f5e-1c79-4ab6-aea4-34806c95946f None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:22.056369 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e8687f5e-1c79-4ab6-aea4-34806c95946f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:22.057504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0bc17-6468-4f5a-9887-7b647913af4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:22.064260 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0bc17-6468-4f5a-9887-7b647913af4b service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:22.066105 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0bc17-6468-4f5a-9887-7b647913af4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:22.066824 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0bc17-6468-4f5a-9887-7b647913af4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:22.072554 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-54e0bc17-6468-4f5a-9887-7b647913af4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:22.249632 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 949/1895] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:22.262577 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66768ead-6401-4f44-a45f-4838f88e5098 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:22.262917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66768ead-6401-4f44-a45f-4838f88e5098 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:22.263410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66768ead-6401-4f44-a45f-4838f88e5098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:22.298000 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0bc17-6468-4f5a-9887-7b647913af4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:22.299569 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0bc17-6468-4f5a-9887-7b647913af4b service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:22.305827 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-66768ead-6401-4f44-a45f-4838f88e5098 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:22.307137 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-66768ead-6401-4f44-a45f-4838f88e5098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:22.327192 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 947/1896] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:47:22.349884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 950/1897] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:47:22.355575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0bc17-6468-4f5a-9887-7b647913af4b service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:22.394818 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0bc17-6468-4f5a-9887-7b647913af4b service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:22.413336 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c61f4-4862-497e-80da-c577b75f10d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:22.413753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c61f4-4862-497e-80da-c577b75f10d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:22.413971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c61f4-4862-497e-80da-c577b75f10d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:22.414290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c61f4-4862-497e-80da-c577b75f10d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:22.415012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c61f4-4862-497e-80da-c577b75f10d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:47:22.415327 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c61f4-4862-497e-80da-c577b75f10d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:22.424634 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c61f4-4862-497e-80da-c577b75f10d2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:22.462930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:22.479550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 948/1898] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:47:22.484242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66768ead-6401-4f44-a45f-4838f88e5098 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:22.500431 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66768ead-6401-4f44-a45f-4838f88e5098 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:22.519113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad074ba-16f0-4eec-b8bd-63b767940d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:22.520069 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad074ba-16f0-4eec-b8bd-63b767940d9f admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:22.524152 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad074ba-16f0-4eec-b8bd-63b767940d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:22.526058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad074ba-16f0-4eec-b8bd-63b767940d9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:22.527243 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad074ba-16f0-4eec-b8bd-63b767940d9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:22.537601 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad074ba-16f0-4eec-b8bd-63b767940d9f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:22.569048 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:22.764983 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c61f4-4862-497e-80da-c577b75f10d2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:22.771776 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 951/1899] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:47:22 2022] POST /identity/v3/projects => generated 373 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:22.777930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c61f4-4862-497e-80da-c577b75f10d2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:23.017356 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669774a1-05da-4576-9b8e-196a3806a637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:23.019349 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669774a1-05da-4576-9b8e-196a3806a637 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:23.019643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669774a1-05da-4576-9b8e-196a3806a637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:23.019955 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669774a1-05da-4576-9b8e-196a3806a637 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:23.023273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669774a1-05da-4576-9b8e-196a3806a637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:23.026938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669774a1-05da-4576-9b8e-196a3806a637 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:23.173844 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 949/1900] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:47:22 2022] POST /identity/v3/users => generated 287 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:47:23.175160 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad074ba-16f0-4eec-b8bd-63b767940d9f admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:23.186931 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad074ba-16f0-4eec-b8bd-63b767940d9f admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:23.192700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5bc6b-230c-4833-8460-421e8d80e9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:23.193683 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5bc6b-230c-4833-8460-421e8d80e9e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:23.194135 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5bc6b-230c-4833-8460-421e8d80e9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:23.194576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5bc6b-230c-4833-8460-421e8d80e9e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:23.195578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5bc6b-230c-4833-8460-421e8d80e9e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:23.211393 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5bc6b-230c-4833-8460-421e8d80e9e7 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:23.242313 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 952/1901] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:23.249545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669774a1-05da-4576-9b8e-196a3806a637 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:23.264126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669774a1-05da-4576-9b8e-196a3806a637 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:23.272403 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651f183-7f89-48c3-8032-b8c6e945fdcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:23.273470 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651f183-7f89-48c3-8032-b8c6e945fdcb admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:23.273928 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651f183-7f89-48c3-8032-b8c6e945fdcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:23.274855 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651f183-7f89-48c3-8032-b8c6e945fdcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:23.276432 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651f183-7f89-48c3-8032-b8c6e945fdcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:23.280644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651f183-7f89-48c3-8032-b8c6e945fdcb admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:23.302401 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 950/1902] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:23.309884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5bc6b-230c-4833-8460-421e8d80e9e7 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:23.333804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5bc6b-230c-4833-8460-421e8d80e9e7 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:23.350703 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b57c1-54b4-4448-96bd-12bf2f87b2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:23.351986 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b57c1-54b4-4448-96bd-12bf2f87b2b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:23.352512 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b57c1-54b4-4448-96bd-12bf2f87b2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:23.352938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b57c1-54b4-4448-96bd-12bf2f87b2b4 admin admin] PATH_INFO: `/v3/projects/4f9e24761e274293abd9f9cb362bb4a3/users/023824b9e78242bd9d10aeb3a52c6d14/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:23.392979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b57c1-54b4-4448-96bd-12bf2f87b2b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f9e24761e274293abd9f9cb362bb4a3, user_id=023824b9e78242bd9d10aeb3a52c6d14, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:23.399062 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b57c1-54b4-4448-96bd-12bf2f87b2b4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:23.506537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 953/1903] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:23 2022] PUT /identity/v3/projects/4f9e24761e274293abd9f9cb362bb4a3/users/023824b9e78242bd9d10aeb3a52c6d14/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:23.512935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56c7e2-2132-4079-b36b-67473727df2f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:23.513401 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56c7e2-2132-4079-b36b-67473727df2f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:23.513649 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56c7e2-2132-4079-b36b-67473727df2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:23.537629 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-dd56c7e2-2132-4079-b36b-67473727df2f None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:23.538889 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-dd56c7e2-2132-4079-b36b-67473727df2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:23.643032 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 951/1904] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:23.655095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b57c1-54b4-4448-96bd-12bf2f87b2b4 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:23.657066 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8c1b57c1-54b4-4448-96bd-12bf2f87b2b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:23.750918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b57c1-54b4-4448-96bd-12bf2f87b2b4 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:23.759300 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a568008-dbdb-49c6-a36b-687414b04604 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:23.760278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a568008-dbdb-49c6-a36b-687414b04604 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:23.760745 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a568008-dbdb-49c6-a36b-687414b04604 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:23.761592 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a568008-dbdb-49c6-a36b-687414b04604 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:23.780533 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a568008-dbdb-49c6-a36b-687414b04604 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:23.783449 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a568008-dbdb-49c6-a36b-687414b04604 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:23.798485 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 954/1905] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:47:25.445154 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0a839-3ac0-4d72-9ff0-b38aed9976d2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:25.445701 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0a839-3ac0-4d72-9ff0-b38aed9976d2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:25.445792 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0a839-3ac0-4d72-9ff0-b38aed9976d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:25.476599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ded0a839-3ac0-4d72-9ff0-b38aed9976d2 None None] MFA Rules not processed for user `023824b9e78242bd9d10aeb3a52c6d14`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:25.477835 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ded0a839-3ac0-4d72-9ff0-b38aed9976d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:25.581796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 952/1906] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:25.585775 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a568008-dbdb-49c6-a36b-687414b04604 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:25.611006 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a568008-dbdb-49c6-a36b-687414b04604 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:25.620301 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ac587-b517-444a-9c49-13b871925b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:25.621086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ac587-b517-444a-9c49-13b871925b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:25.621447 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ac587-b517-444a-9c49-13b871925b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:25.621812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ac587-b517-444a-9c49-13b871925b50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:25.622905 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ac587-b517-444a-9c49-13b871925b50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:25.625088 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ac587-b517-444a-9c49-13b871925b50 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:25.742980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 955/1907] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:47:25 2022] POST /identity/v3/projects => generated 371 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:25.749977 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0a839-3ac0-4d72-9ff0-b38aed9976d2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:25.770850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0a839-3ac0-4d72-9ff0-b38aed9976d2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:25.778857 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:25.842036 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2fbec-29cc-45b2-90b2-7fd7f427094d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:25.842736 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2fbec-29cc-45b2-90b2-7fd7f427094d admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:25.843024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2fbec-29cc-45b2-90b2-7fd7f427094d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:25.843294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2fbec-29cc-45b2-90b2-7fd7f427094d admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:25.844035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2fbec-29cc-45b2-90b2-7fd7f427094d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:25.846038 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2fbec-29cc-45b2-90b2-7fd7f427094d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:25.946152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 953/1908] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:47:25 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:25.951182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ac587-b517-444a-9c49-13b871925b50 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:25.963087 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ac587-b517-444a-9c49-13b871925b50 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:25.969304 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c596228-513e-4223-8afa-5e91fa615d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:25.970065 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c596228-513e-4223-8afa-5e91fa615d31 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:25.970346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c596228-513e-4223-8afa-5e91fa615d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:25.970627 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c596228-513e-4223-8afa-5e91fa615d31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:25.971540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c596228-513e-4223-8afa-5e91fa615d31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:25.973239 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c596228-513e-4223-8afa-5e91fa615d31 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:25.985547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 956/1909] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:25.991813 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2fbec-29cc-45b2-90b2-7fd7f427094d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:26.005178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2fbec-29cc-45b2-90b2-7fd7f427094d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:26.010188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0caec6e-833c-45ad-8914-a472cfd4938a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:26.011101 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0caec6e-833c-45ad-8914-a472cfd4938a admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:26.011377 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0caec6e-833c-45ad-8914-a472cfd4938a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:26.011654 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0caec6e-833c-45ad-8914-a472cfd4938a admin admin] PATH_INFO: `/v3/projects/ab6ce9a4b32f4edfbd9c40da74938006/users/80f62eadb2404a74bdffb4adba33f1f7/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:26.031770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0caec6e-833c-45ad-8914-a472cfd4938a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab6ce9a4b32f4edfbd9c40da74938006, user_id=80f62eadb2404a74bdffb4adba33f1f7, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:26.034540 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0caec6e-833c-45ad-8914-a472cfd4938a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:26.102274 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 954/1910] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:25 2022] PUT /identity/v3/projects/ab6ce9a4b32f4edfbd9c40da74938006/users/80f62eadb2404a74bdffb4adba33f1f7/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:26.106446 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c596228-513e-4223-8afa-5e91fa615d31 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:26.117729 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c596228-513e-4223-8afa-5e91fa615d31 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:26.182866 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1506d0-99f9-49af-bd79-6f509ddba6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:26.183539 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1506d0-99f9-49af-bd79-6f509ddba6d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:26.183854 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1506d0-99f9-49af-bd79-6f509ddba6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:26.184339 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1506d0-99f9-49af-bd79-6f509ddba6d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:26.185327 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1506d0-99f9-49af-bd79-6f509ddba6d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:26.186921 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1506d0-99f9-49af-bd79-6f509ddba6d6 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:26.201144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 957/1911] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:26.204883 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0caec6e-833c-45ad-8914-a472cfd4938a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:26.217451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0caec6e-833c-45ad-8914-a472cfd4938a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:26.224759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11d56f-a7eb-446a-aaca-3896916c8be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:26.230258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11d56f-a7eb-446a-aaca-3896916c8be8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:26.230258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11d56f-a7eb-446a-aaca-3896916c8be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:26.230258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11d56f-a7eb-446a-aaca-3896916c8be8 admin admin] PATH_INFO: `/v3/projects/ab6ce9a4b32f4edfbd9c40da74938006/users/80f62eadb2404a74bdffb4adba33f1f7/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:26.230258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11d56f-a7eb-446a-aaca-3896916c8be8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab6ce9a4b32f4edfbd9c40da74938006, user_id=80f62eadb2404a74bdffb4adba33f1f7, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:26.231812 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11d56f-a7eb-446a-aaca-3896916c8be8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:26.244605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 955/1912] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:26 2022] PUT /identity/v3/projects/ab6ce9a4b32f4edfbd9c40da74938006/users/80f62eadb2404a74bdffb4adba33f1f7/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:28.239754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fec9a8-c75e-4fcb-8e1e-f95d3671866b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:28.240773 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fec9a8-c75e-4fcb-8e1e-f95d3671866b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:28.241060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fec9a8-c75e-4fcb-8e1e-f95d3671866b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:28.275281 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c0fec9a8-c75e-4fcb-8e1e-f95d3671866b None None] MFA Rules not processed for user `80f62eadb2404a74bdffb4adba33f1f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:28.280178 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c0fec9a8-c75e-4fcb-8e1e-f95d3671866b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:28.288803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:28.396637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 958/1913] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:28 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:28.479430 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11d56f-a7eb-446a-aaca-3896916c8be8 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:28.482691 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9b11d56f-a7eb-446a-aaca-3896916c8be8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:28.633155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11d56f-a7eb-446a-aaca-3896916c8be8 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:28.639498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41558b5-3d54-4346-98d5-9759f89b18fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:28.643399 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41558b5-3d54-4346-98d5-9759f89b18fe service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:28.643399 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41558b5-3d54-4346-98d5-9759f89b18fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:28.643399 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41558b5-3d54-4346-98d5-9759f89b18fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:28.741442 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41558b5-3d54-4346-98d5-9759f89b18fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:28.744374 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41558b5-3d54-4346-98d5-9759f89b18fe service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:28.773783 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 956/1914] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:47:31.021905 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fec9a8-c75e-4fcb-8e1e-f95d3671866b None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:31.034770 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fec9a8-c75e-4fcb-8e1e-f95d3671866b None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:31.039058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe49b5-6444-47ab-b3a1-c9301b615c74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:31.039767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe49b5-6444-47ab-b3a1-c9301b615c74 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:31.040056 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe49b5-6444-47ab-b3a1-c9301b615c74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:31.040334 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe49b5-6444-47ab-b3a1-c9301b615c74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:31.051564 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe49b5-6444-47ab-b3a1-c9301b615c74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:31.053075 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe49b5-6444-47ab-b3a1-c9301b615c74 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:31.064282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 959/1915] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:31 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:47:32.822863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41558b5-3d54-4346-98d5-9759f89b18fe service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:32.828296 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a41558b5-3d54-4346-98d5-9759f89b18fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:32.877904 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:32.979354 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41558b5-3d54-4346-98d5-9759f89b18fe service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:32.997835 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1df96-d569-448d-9d46-99ba80e927b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:32.999362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1df96-d569-448d-9d46-99ba80e927b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:32.999959 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1df96-d569-448d-9d46-99ba80e927b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:33.000468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1df96-d569-448d-9d46-99ba80e927b2 admin admin] PATH_INFO: `/v3/users/523248ec76614252a9811b4f1595e1b9` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:33.002957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1df96-d569-448d-9d46-99ba80e927b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=523248ec76614252a9811b4f1595e1b9)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:33.007499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1df96-d569-448d-9d46-99ba80e927b2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:33.396674 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9ce1df96-d569-448d-9d46-99ba80e927b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '523248ec76614252a9811b4f1595e1b9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:33.397619 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9ce1df96-d569-448d-9d46-99ba80e927b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '523248ec76614252a9811b4f1595e1b9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:33.398428 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9ce1df96-d569-448d-9d46-99ba80e927b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '523248ec76614252a9811b4f1595e1b9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:33.414843 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9ce1df96-d569-448d-9d46-99ba80e927b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '523248ec76614252a9811b4f1595e1b9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:33.424474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9ce1df96-d569-448d-9d46-99ba80e927b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '523248ec76614252a9811b4f1595e1b9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:33.548687 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 957/1916] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:32 2022] DELETE /identity/v3/users/523248ec76614252a9811b4f1595e1b9 => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:33.552846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2678c-ef02-4b44-a27c-4670888a3565 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:33.553400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2678c-ef02-4b44-a27c-4670888a3565 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:33.554341 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2678c-ef02-4b44-a27c-4670888a3565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:33.629657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-53a2678c-ef02-4b44-a27c-4670888a3565 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:33.631267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-53a2678c-ef02-4b44-a27c-4670888a3565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:33.739100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1df96-d569-448d-9d46-99ba80e927b2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:33.741189 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9ce1df96-d569-448d-9d46-99ba80e927b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:33.753380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 960/1917] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:33.771633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2678c-ef02-4b44-a27c-4670888a3565 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:33.781976 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-53a2678c-ef02-4b44-a27c-4670888a3565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:33.795316 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1df96-d569-448d-9d46-99ba80e927b2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:33.818495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81116fbd-e2bf-4484-b241-eb3fcdb4b93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:33.820058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81116fbd-e2bf-4484-b241-eb3fcdb4b93f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:33.820546 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81116fbd-e2bf-4484-b241-eb3fcdb4b93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:33.821004 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81116fbd-e2bf-4484-b241-eb3fcdb4b93f admin admin] PATH_INFO: `/v3/users/da50b3d5815e4e9e9f6c4a3d36da071c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:33.828042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81116fbd-e2bf-4484-b241-eb3fcdb4b93f admin admin] RBAC: Authorizing `identity:delete_user(user_id=da50b3d5815e4e9e9f6c4a3d36da071c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:33.838495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81116fbd-e2bf-4484-b241-eb3fcdb4b93f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:33.976906 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2678c-ef02-4b44-a27c-4670888a3565 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:33.982503 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a19d8-c4dc-412c-a3fa-10bfb4d8041e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:33.983171 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a19d8-c4dc-412c-a3fa-10bfb4d8041e service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:33.983485 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a19d8-c4dc-412c-a3fa-10bfb4d8041e service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:33.983854 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a19d8-c4dc-412c-a3fa-10bfb4d8041e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:33.996824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a19d8-c4dc-412c-a3fa-10bfb4d8041e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:33.998396 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a19d8-c4dc-412c-a3fa-10bfb4d8041e service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:34.023992 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 961/1918] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:47:34.130697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81116fbd-e2bf-4484-b241-eb3fcdb4b93f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da50b3d5815e4e9e9f6c4a3d36da071c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:34.132007 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81116fbd-e2bf-4484-b241-eb3fcdb4b93f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da50b3d5815e4e9e9f6c4a3d36da071c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:34.133931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81116fbd-e2bf-4484-b241-eb3fcdb4b93f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da50b3d5815e4e9e9f6c4a3d36da071c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:34.161184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81116fbd-e2bf-4484-b241-eb3fcdb4b93f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da50b3d5815e4e9e9f6c4a3d36da071c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:34.176844 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81116fbd-e2bf-4484-b241-eb3fcdb4b93f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da50b3d5815e4e9e9f6c4a3d36da071c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:34.272455 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 958/1919] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:33 2022] DELETE /identity/v3/users/da50b3d5815e4e9e9f6c4a3d36da071c => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:34.278674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554b8d5-d103-4bbb-844a-547ffcde0441 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:34.279524 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554b8d5-d103-4bbb-844a-547ffcde0441 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:34.280054 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554b8d5-d103-4bbb-844a-547ffcde0441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:34.314456 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6554b8d5-d103-4bbb-844a-547ffcde0441 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:34.317069 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6554b8d5-d103-4bbb-844a-547ffcde0441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:34.384140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:34.437604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 962/1920] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:34.463966 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81116fbd-e2bf-4484-b241-eb3fcdb4b93f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:34.467851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-81116fbd-e2bf-4484-b241-eb3fcdb4b93f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:34.490646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:34.585808 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81116fbd-e2bf-4484-b241-eb3fcdb4b93f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:34.590034 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a081f-cf8f-49db-9f1e-fb00718e80df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:34.591429 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a081f-cf8f-49db-9f1e-fb00718e80df service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:34.591843 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a081f-cf8f-49db-9f1e-fb00718e80df service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:34.592127 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a081f-cf8f-49db-9f1e-fb00718e80df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:34.603121 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a081f-cf8f-49db-9f1e-fb00718e80df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:34.604803 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a081f-cf8f-49db-9f1e-fb00718e80df service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:34.619078 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 959/1921] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:47:34.718741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554b8d5-d103-4bbb-844a-547ffcde0441 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:34.721996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6554b8d5-d103-4bbb-844a-547ffcde0441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:34.748212 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554b8d5-d103-4bbb-844a-547ffcde0441 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:34.757198 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940ca3f-a1a9-4202-a855-35aa63979cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:34.758272 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940ca3f-a1a9-4202-a855-35aa63979cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:34.758772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940ca3f-a1a9-4202-a855-35aa63979cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:34.758918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940ca3f-a1a9-4202-a855-35aa63979cc2 admin admin] PATH_INFO: `/v3/projects/a8e1978e517b49678fbe4d79583eac10` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:34.760592 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5940ca3f-a1a9-4202-a855-35aa63979cc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8e1978e517b49678fbe4d79583eac10)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:34.767093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5940ca3f-a1a9-4202-a855-35aa63979cc2 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:34.914688 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5940ca3f-a1a9-4202-a855-35aa63979cc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8e1978e517b49678fbe4d79583eac10', 'request_id': 'req-5940ca3f-a1a9-4202-a855-35aa63979cc2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:34.924260 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5940ca3f-a1a9-4202-a855-35aa63979cc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8e1978e517b49678fbe4d79583eac10', 'request_id': 'req-5940ca3f-a1a9-4202-a855-35aa63979cc2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:34.932296 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5940ca3f-a1a9-4202-a855-35aa63979cc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8e1978e517b49678fbe4d79583eac10', 'request_id': 'req-5940ca3f-a1a9-4202-a855-35aa63979cc2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:35.009591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5940ca3f-a1a9-4202-a855-35aa63979cc2 admin admin] The token cache is being invalidate because project a8e1978e517b49678fbe4d79583eac10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:35.011339 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5940ca3f-a1a9-4202-a855-35aa63979cc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:35.011339 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5940ca3f-a1a9-4202-a855-35aa63979cc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:35.013374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 963/1922] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:34 2022] DELETE /identity/v3/projects/a8e1978e517b49678fbe4d79583eac10 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:35.019801 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a081f-cf8f-49db-9f1e-fb00718e80df service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:35.035502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a081f-cf8f-49db-9f1e-fb00718e80df service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:35.114452 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a074e6-b3c3-4242-9e41-04ef06659c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:35.115476 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a074e6-b3c3-4242-9e41-04ef06659c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:35.116183 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a074e6-b3c3-4242-9e41-04ef06659c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:35.116576 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a074e6-b3c3-4242-9e41-04ef06659c2c admin admin] PATH_INFO: `/v3/users/5dfc5fe8cd5e4f388b50b1fb0ca6687a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:35.118266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a074e6-b3c3-4242-9e41-04ef06659c2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dfc5fe8cd5e4f388b50b1fb0ca6687a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:35.120840 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a074e6-b3c3-4242-9e41-04ef06659c2c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:35.401006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d4a074e6-b3c3-4242-9e41-04ef06659c2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dfc5fe8cd5e4f388b50b1fb0ca6687a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:35.402651 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d4a074e6-b3c3-4242-9e41-04ef06659c2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dfc5fe8cd5e4f388b50b1fb0ca6687a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:35.404597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d4a074e6-b3c3-4242-9e41-04ef06659c2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dfc5fe8cd5e4f388b50b1fb0ca6687a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:35.456320 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d4a074e6-b3c3-4242-9e41-04ef06659c2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dfc5fe8cd5e4f388b50b1fb0ca6687a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:35.480897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d4a074e6-b3c3-4242-9e41-04ef06659c2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dfc5fe8cd5e4f388b50b1fb0ca6687a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:35.625946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 960/1923] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:35 2022] DELETE /identity/v3/users/5dfc5fe8cd5e4f388b50b1fb0ca6687a => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:35.700007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940ca3f-a1a9-4202-a855-35aa63979cc2 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:35.703641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5940ca3f-a1a9-4202-a855-35aa63979cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:35.743089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940ca3f-a1a9-4202-a855-35aa63979cc2 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:35.880618 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f9759-4a9e-4f49-9103-1e113fa8253e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:35.883906 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f9759-4a9e-4f49-9103-1e113fa8253e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:35.883906 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f9759-4a9e-4f49-9103-1e113fa8253e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:35.883906 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f9759-4a9e-4f49-9103-1e113fa8253e admin admin] PATH_INFO: `/v3/projects/bf93e90ca0364aebb603729b54a3b84e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:35.884721 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f9759-4a9e-4f49-9103-1e113fa8253e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf93e90ca0364aebb603729b54a3b84e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:35.888620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f9759-4a9e-4f49-9103-1e113fa8253e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:36.080907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-af5f9759-4a9e-4f49-9103-1e113fa8253e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf93e90ca0364aebb603729b54a3b84e', 'request_id': 'req-af5f9759-4a9e-4f49-9103-1e113fa8253e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:36.092351 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-af5f9759-4a9e-4f49-9103-1e113fa8253e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf93e90ca0364aebb603729b54a3b84e', 'request_id': 'req-af5f9759-4a9e-4f49-9103-1e113fa8253e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:36.095230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:36.102474 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-af5f9759-4a9e-4f49-9103-1e113fa8253e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf93e90ca0364aebb603729b54a3b84e', 'request_id': 'req-af5f9759-4a9e-4f49-9103-1e113fa8253e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:36.218357 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-af5f9759-4a9e-4f49-9103-1e113fa8253e admin admin] The token cache is being invalidate because project bf93e90ca0364aebb603729b54a3b84e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:36.219122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-af5f9759-4a9e-4f49-9103-1e113fa8253e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:36.220897 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-af5f9759-4a9e-4f49-9103-1e113fa8253e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:36.222521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 964/1924] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:35 2022] DELETE /identity/v3/projects/bf93e90ca0364aebb603729b54a3b84e => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:36.228794 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a074e6-b3c3-4242-9e41-04ef06659c2c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:36.231348 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d4a074e6-b3c3-4242-9e41-04ef06659c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:36.263898 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a074e6-b3c3-4242-9e41-04ef06659c2c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:36.269169 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f9759-4a9e-4f49-9103-1e113fa8253e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:36.273183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-af5f9759-4a9e-4f49-9103-1e113fa8253e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:36.328884 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:36.402843 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd9f95-7033-4d29-93f7-45e11c7b0d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:36.405828 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd9f95-7033-4d29-93f7-45e11c7b0d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:36.406590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd9f95-7033-4d29-93f7-45e11c7b0d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:36.407369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd9f95-7033-4d29-93f7-45e11c7b0d51 admin admin] PATH_INFO: `/v3/users/d2b09cb6e12c47338d73df57982a6376` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:36.410637 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd9f95-7033-4d29-93f7-45e11c7b0d51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2b09cb6e12c47338d73df57982a6376)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:36.420385 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f9759-4a9e-4f49-9103-1e113fa8253e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:36.423732 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd9f95-7033-4d29-93f7-45e11c7b0d51 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:36.429415 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4295a-889a-49ed-9b8d-63f72041f556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:36.431615 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4295a-889a-49ed-9b8d-63f72041f556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:36.432542 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4295a-889a-49ed-9b8d-63f72041f556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:36.433301 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4295a-889a-49ed-9b8d-63f72041f556 admin admin] PATH_INFO: `/v3/projects/5eaaccb93ff34bad9d6f4fa4bc3f24e4` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:36.435961 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4295a-889a-49ed-9b8d-63f72041f556 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eaaccb93ff34bad9d6f4fa4bc3f24e4)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:36.440520 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4295a-889a-49ed-9b8d-63f72041f556 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:36.728562 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c7b4295a-889a-49ed-9b8d-63f72041f556 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eaaccb93ff34bad9d6f4fa4bc3f24e4', 'request_id': 'req-c7b4295a-889a-49ed-9b8d-63f72041f556'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:36.737531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d1cd9f95-7033-4d29-93f7-45e11c7b0d51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2b09cb6e12c47338d73df57982a6376'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:36.737886 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c7b4295a-889a-49ed-9b8d-63f72041f556 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eaaccb93ff34bad9d6f4fa4bc3f24e4', 'request_id': 'req-c7b4295a-889a-49ed-9b8d-63f72041f556'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:36.738497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d1cd9f95-7033-4d29-93f7-45e11c7b0d51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2b09cb6e12c47338d73df57982a6376'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:36.739239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d1cd9f95-7033-4d29-93f7-45e11c7b0d51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2b09cb6e12c47338d73df57982a6376'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:36.751021 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c7b4295a-889a-49ed-9b8d-63f72041f556 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eaaccb93ff34bad9d6f4fa4bc3f24e4', 'request_id': 'req-c7b4295a-889a-49ed-9b8d-63f72041f556'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:36.758498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d1cd9f95-7033-4d29-93f7-45e11c7b0d51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2b09cb6e12c47338d73df57982a6376'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:36.768466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-d1cd9f95-7033-4d29-93f7-45e11c7b0d51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2b09cb6e12c47338d73df57982a6376'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:36.852863 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c7b4295a-889a-49ed-9b8d-63f72041f556 admin admin] The token cache is being invalidate because project 5eaaccb93ff34bad9d6f4fa4bc3f24e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:36.852863 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c7b4295a-889a-49ed-9b8d-63f72041f556 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:36.852863 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c7b4295a-889a-49ed-9b8d-63f72041f556 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:36.855609 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 965/1925] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:36 2022] DELETE /identity/v3/projects/5eaaccb93ff34bad9d6f4fa4bc3f24e4 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:36.864775 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac6e19-f907-474c-af40-d38c80010d3b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:36.865275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac6e19-f907-474c-af40-d38c80010d3b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:36.865915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac6e19-f907-474c-af40-d38c80010d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:36.897645 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-f1ac6e19-f907-474c-af40-d38c80010d3b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:36.900092 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f1ac6e19-f907-474c-af40-d38c80010d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:36.931263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 961/1926] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:36 2022] DELETE /identity/v3/users/d2b09cb6e12c47338d73df57982a6376 => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:36.982801 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 966/1927] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:36.992483 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6ee45-6959-4dbd-b856-42e492767b71 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:36.993071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6ee45-6959-4dbd-b856-42e492767b71 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:36.993558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6ee45-6959-4dbd-b856-42e492767b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:37.025963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d8f6ee45-6959-4dbd-b856-42e492767b71 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:37.029720 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d8f6ee45-6959-4dbd-b856-42e492767b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:37.154616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 962/1928] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:37.159206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac6e19-f907-474c-af40-d38c80010d3b None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:37.181994 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac6e19-f907-474c-af40-d38c80010d3b None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:37.189500 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f4767-6ad9-4723-b364-d4b09c9bcd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:37.190524 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f4767-6ad9-4723-b364-d4b09c9bcd12 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:37.190921 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f4767-6ad9-4723-b364-d4b09c9bcd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:37.191392 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f4767-6ad9-4723-b364-d4b09c9bcd12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:37.192762 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f4767-6ad9-4723-b364-d4b09c9bcd12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:47:37.193176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f4767-6ad9-4723-b364-d4b09c9bcd12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:37.197902 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f4767-6ad9-4723-b364-d4b09c9bcd12 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:37.224570 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 967/1929] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:47:37.225881 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6ee45-6959-4dbd-b856-42e492767b71 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:37.236320 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6ee45-6959-4dbd-b856-42e492767b71 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:37.241468 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870610c-6565-4ca0-b7ba-e8c324d69ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:37.245949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870610c-6565-4ca0-b7ba-e8c324d69ceb admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:37.245949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870610c-6565-4ca0-b7ba-e8c324d69ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:37.245949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870610c-6565-4ca0-b7ba-e8c324d69ceb admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:37.245949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870610c-6565-4ca0-b7ba-e8c324d69ceb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:37.248980 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870610c-6565-4ca0-b7ba-e8c324d69ceb admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:37.399821 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 963/1930] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:47:37 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:37.404755 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f4767-6ad9-4723-b364-d4b09c9bcd12 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:37.418542 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f4767-6ad9-4723-b364-d4b09c9bcd12 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:37.427205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:37.513130 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29068d5f-07fb-4f63-a14d-ac1186614463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:37.513856 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29068d5f-07fb-4f63-a14d-ac1186614463 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:37.514133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29068d5f-07fb-4f63-a14d-ac1186614463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:37.514411 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29068d5f-07fb-4f63-a14d-ac1186614463 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:37.515195 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29068d5f-07fb-4f63-a14d-ac1186614463 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:37.517448 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29068d5f-07fb-4f63-a14d-ac1186614463 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:37.630620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 968/1931] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:47:37 2022] POST /identity/v3/users => generated 293 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:37.635508 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870610c-6565-4ca0-b7ba-e8c324d69ceb admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:37.650832 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870610c-6565-4ca0-b7ba-e8c324d69ceb admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:37.656923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15163adb-fcde-4d23-b2aa-1058fa1a7a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:37.657212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15163adb-fcde-4d23-b2aa-1058fa1a7a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:37.657627 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15163adb-fcde-4d23-b2aa-1058fa1a7a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:37.657956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15163adb-fcde-4d23-b2aa-1058fa1a7a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:37.658785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15163adb-fcde-4d23-b2aa-1058fa1a7a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:37.660680 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15163adb-fcde-4d23-b2aa-1058fa1a7a1a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:37.680276 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29068d5f-07fb-4f63-a14d-ac1186614463 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:37.688177 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-29068d5f-07fb-4f63-a14d-ac1186614463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:37.688343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 964/1932] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:37.690751 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15163adb-fcde-4d23-b2aa-1058fa1a7a1a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:37.704257 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29068d5f-07fb-4f63-a14d-ac1186614463 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:37.724326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15163adb-fcde-4d23-b2aa-1058fa1a7a1a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:37.726028 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522b2a8-de3d-4de2-b64a-44924e16a97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:37.727225 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522b2a8-de3d-4de2-b64a-44924e16a97c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:37.727797 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522b2a8-de3d-4de2-b64a-44924e16a97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:37.728299 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522b2a8-de3d-4de2-b64a-44924e16a97c admin admin] PATH_INFO: `/v3/projects/c747b26e490d4910ad598f7b58efe530` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:37.729900 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522b2a8-de3d-4de2-b64a-44924e16a97c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c747b26e490d4910ad598f7b58efe530)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:37.733268 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522b2a8-de3d-4de2-b64a-44924e16a97c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:37.736857 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd87c86-9465-42a5-8713-ff10ac9976d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:37.738919 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd87c86-9465-42a5-8713-ff10ac9976d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:37.739427 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd87c86-9465-42a5-8713-ff10ac9976d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:37.739985 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd87c86-9465-42a5-8713-ff10ac9976d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:37.741322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd87c86-9465-42a5-8713-ff10ac9976d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:37.744533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd87c86-9465-42a5-8713-ff10ac9976d8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:37.770174 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 965/1933] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:37.775813 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd87c86-9465-42a5-8713-ff10ac9976d8 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:37.792897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:37.803081 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd87c86-9465-42a5-8713-ff10ac9976d8 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:37.812291 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8f507-9c4f-4292-8e0d-3c70a56c2fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:37.813974 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8f507-9c4f-4292-8e0d-3c70a56c2fee admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:37.814476 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8f507-9c4f-4292-8e0d-3c70a56c2fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:37.814977 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8f507-9c4f-4292-8e0d-3c70a56c2fee admin admin] PATH_INFO: `/v3/projects/79d2e618d9cd4648bc71619b974a0fa3/users/77a3bb1f73db42759a61495000300612/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:37.852935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8f507-9c4f-4292-8e0d-3c70a56c2fee admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d2e618d9cd4648bc71619b974a0fa3, user_id=77a3bb1f73db42759a61495000300612, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:37.859042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8f507-9c4f-4292-8e0d-3c70a56c2fee admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:37.977313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 966/1934] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:37 2022] PUT /identity/v3/projects/79d2e618d9cd4648bc71619b974a0fa3/users/77a3bb1f73db42759a61495000300612/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:38.002665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d344f-3f25-4e30-9700-02efded441ff None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:38.002887 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f522b2a8-de3d-4de2-b64a-44924e16a97c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c747b26e490d4910ad598f7b58efe530', 'request_id': 'req-f522b2a8-de3d-4de2-b64a-44924e16a97c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:38.004565 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d344f-3f25-4e30-9700-02efded441ff None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:38.005066 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d344f-3f25-4e30-9700-02efded441ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:38.016778 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f522b2a8-de3d-4de2-b64a-44924e16a97c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c747b26e490d4910ad598f7b58efe530', 'request_id': 'req-f522b2a8-de3d-4de2-b64a-44924e16a97c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:38.032632 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f522b2a8-de3d-4de2-b64a-44924e16a97c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c747b26e490d4910ad598f7b58efe530', 'request_id': 'req-f522b2a8-de3d-4de2-b64a-44924e16a97c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:38.054097 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-642d344f-3f25-4e30-9700-02efded441ff None None] MFA Rules not processed for user `77a3bb1f73db42759a61495000300612`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:38.055417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-642d344f-3f25-4e30-9700-02efded441ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:38.160859 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 967/1935] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:47:38.176912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f522b2a8-de3d-4de2-b64a-44924e16a97c admin admin] The token cache is being invalidate because project c747b26e490d4910ad598f7b58efe530 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:38.177910 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f522b2a8-de3d-4de2-b64a-44924e16a97c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:38.178985 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f522b2a8-de3d-4de2-b64a-44924e16a97c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:38.180011 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d344f-3f25-4e30-9700-02efded441ff None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:38.182460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-642d344f-3f25-4e30-9700-02efded441ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:38.183499 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 969/1936] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:37 2022] DELETE /identity/v3/projects/c747b26e490d4910ad598f7b58efe530 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:38.191073 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751c304-fe36-47ab-ac98-4a59dc99a2b4 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:38.191706 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751c304-fe36-47ab-ac98-4a59dc99a2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:38.191989 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751c304-fe36-47ab-ac98-4a59dc99a2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:38.241257 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-8751c304-fe36-47ab-ac98-4a59dc99a2b4 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:38.247390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8751c304-fe36-47ab-ac98-4a59dc99a2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:38.360353 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d344f-3f25-4e30-9700-02efded441ff None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:38.368692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b8274-c3d0-4d7a-a48b-4de1ba6425b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:38.370179 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b8274-c3d0-4d7a-a48b-4de1ba6425b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:38.370549 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b8274-c3d0-4d7a-a48b-4de1ba6425b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:38.371086 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b8274-c3d0-4d7a-a48b-4de1ba6425b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:38.374622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e51b8274-c3d0-4d7a-a48b-4de1ba6425b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:38.391044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 970/1937] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:47:38.391044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66474b-269a-4fd3-826c-2172dc26f913 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:38.391044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66474b-269a-4fd3-826c-2172dc26f913 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:38.391044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66474b-269a-4fd3-826c-2172dc26f913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:38.415881 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b8274-c3d0-4d7a-a48b-4de1ba6425b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:38.422377 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b8274-c3d0-4d7a-a48b-4de1ba6425b9 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:38.431249 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-ef66474b-269a-4fd3-826c-2172dc26f913 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:38.432625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ef66474b-269a-4fd3-826c-2172dc26f913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:38.494795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 971/1938] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:47:38.497452 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66474b-269a-4fd3-826c-2172dc26f913 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:38.497920 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 968/1939] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:47:38.535744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66474b-269a-4fd3-826c-2172dc26f913 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:38.557909 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c229e-4159-4fe8-b797-fdbb084899c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:38.558790 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c229e-4159-4fe8-b797-fdbb084899c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:38.559216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c229e-4159-4fe8-b797-fdbb084899c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:38.559608 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c229e-4159-4fe8-b797-fdbb084899c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:38.560586 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c229e-4159-4fe8-b797-fdbb084899c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:47:38.560973 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c229e-4159-4fe8-b797-fdbb084899c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:38.562832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c229e-4159-4fe8-b797-fdbb084899c0 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:38.575281 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:38.580693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 972/1940] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:47:38.586422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b8274-c3d0-4d7a-a48b-4de1ba6425b9 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:38.602759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b8274-c3d0-4d7a-a48b-4de1ba6425b9 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:38.619999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c206a8-8d1f-458d-995b-714d9ff408ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:38.621168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c206a8-8d1f-458d-995b-714d9ff408ce admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:38.621648 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c206a8-8d1f-458d-995b-714d9ff408ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:38.622221 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c206a8-8d1f-458d-995b-714d9ff408ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:38.623343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c206a8-8d1f-458d-995b-714d9ff408ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:38.626510 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c206a8-8d1f-458d-995b-714d9ff408ce admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:38.813742 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 969/1941] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:47:38 2022] POST /identity/v3/projects => generated 373 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:38.819038 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c229e-4159-4fe8-b797-fdbb084899c0 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:38.841348 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c229e-4159-4fe8-b797-fdbb084899c0 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:38.856581 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d1f25-4b31-4696-b879-fb9369ee178e None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:38.858184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d1f25-4b31-4696-b879-fb9369ee178e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:38.858888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d1f25-4b31-4696-b879-fb9369ee178e None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:38.862837 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 970/1942] 10.222.0.57 () {54 vars in 921 bytes} [Mon Jul 18 08:47:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:47:38.880478 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a7bf0-388e-47b6-ab8f-24191953123c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:38.881210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a7bf0-388e-47b6-ab8f-24191953123c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:38.881584 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a7bf0-388e-47b6-ab8f-24191953123c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:38.905771 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-047a7bf0-388e-47b6-ab8f-24191953123c None None] MFA Rules not processed for user `d606fd7ef269460a8110223932f02d19`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:38.907143 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-047a7bf0-388e-47b6-ab8f-24191953123c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:38.962519 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6313f97-dadd-43cb-9938-f72279cae5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:38.963596 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6313f97-dadd-43cb-9938-f72279cae5b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:38.963975 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6313f97-dadd-43cb-9938-f72279cae5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:38.964334 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6313f97-dadd-43cb-9938-f72279cae5b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:38.965241 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6313f97-dadd-43cb-9938-f72279cae5b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:38.968997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6313f97-dadd-43cb-9938-f72279cae5b4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:38.983117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:39.011863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 971/1943] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Jul 18 08:47:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:47:39.026197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a7bf0-388e-47b6-ab8f-24191953123c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:39.028591 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-047a7bf0-388e-47b6-ab8f-24191953123c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:39.042080 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a7bf0-388e-47b6-ab8f-24191953123c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:39.048655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfffe77-023a-4f43-87a5-8642fcab8d83 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd606fd7ef269460a8110223932f02d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:39.049325 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfffe77-023a-4f43-87a5-8642fcab8d83 service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:39.049600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfffe77-023a-4f43-87a5-8642fcab8d83 service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:39.049890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfffe77-023a-4f43-87a5-8642fcab8d83 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:39.059971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfffe77-023a-4f43-87a5-8642fcab8d83 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:39.061480 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfffe77-023a-4f43-87a5-8642fcab8d83 service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:39.072836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 972/1944] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:47:39.098764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfffe77-023a-4f43-87a5-8642fcab8d83 service placement] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:39.100466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6cfffe77-023a-4f43-87a5-8642fcab8d83 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:39.130086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 973/1945] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:47:38 2022] POST /identity/v3/users => generated 287 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:47:39.135623 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6313f97-dadd-43cb-9938-f72279cae5b4 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:39.143630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfffe77-023a-4f43-87a5-8642fcab8d83 service placement] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:39.151369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243b8eb-9b09-4895-82b1-fd3be54f58bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:39.153657 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243b8eb-9b09-4895-82b1-fd3be54f58bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:39.154527 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243b8eb-9b09-4895-82b1-fd3be54f58bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:39.154997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243b8eb-9b09-4895-82b1-fd3be54f58bf admin admin] PATH_INFO: `/v3/users/67a5cc6144c0434b9f16fb050c4c7662` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:39.158412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243b8eb-9b09-4895-82b1-fd3be54f58bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=67a5cc6144c0434b9f16fb050c4c7662)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:39.162454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243b8eb-9b09-4895-82b1-fd3be54f58bf admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:39.165711 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6313f97-dadd-43cb-9938-f72279cae5b4 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:39.170462 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e304e92-0715-4517-8789-ed4905998a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:39.171734 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e304e92-0715-4517-8789-ed4905998a36 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:39.172217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e304e92-0715-4517-8789-ed4905998a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:39.172665 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e304e92-0715-4517-8789-ed4905998a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:39.173506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e304e92-0715-4517-8789-ed4905998a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:39.175328 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e304e92-0715-4517-8789-ed4905998a36 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:39.187431 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 974/1946] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:39.191980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e304e92-0715-4517-8789-ed4905998a36 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:39.206295 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e304e92-0715-4517-8789-ed4905998a36 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:39.213967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f79ad0-6331-4347-b476-2f279835df95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:39.214827 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f79ad0-6331-4347-b476-2f279835df95 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:39.215269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f79ad0-6331-4347-b476-2f279835df95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:39.216250 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f79ad0-6331-4347-b476-2f279835df95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:39.217461 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f79ad0-6331-4347-b476-2f279835df95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:39.219259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f79ad0-6331-4347-b476-2f279835df95 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:39.230909 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 975/1947] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:39.236074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f79ad0-6331-4347-b476-2f279835df95 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:39.252405 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f79ad0-6331-4347-b476-2f279835df95 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:39.257475 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71f292-abed-46b2-8aac-303631315a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:39.258419 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71f292-abed-46b2-8aac-303631315a19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:39.258840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71f292-abed-46b2-8aac-303631315a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:39.259139 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71f292-abed-46b2-8aac-303631315a19 admin admin] PATH_INFO: `/v3/projects/2583fb3db2dc45a297d10d0e71db6f51/users/166426445c654182a19d583b24242536/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:39.279557 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d71f292-abed-46b2-8aac-303631315a19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2583fb3db2dc45a297d10d0e71db6f51, user_id=166426445c654182a19d583b24242536, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:39.282514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d71f292-abed-46b2-8aac-303631315a19 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:39.386391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 976/1948] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:39 2022] PUT /identity/v3/projects/2583fb3db2dc45a297d10d0e71db6f51/users/166426445c654182a19d583b24242536/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:39.411738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89014274-0c1e-46a6-971c-cf0b204725cc None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:39.411738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89014274-0c1e-46a6-971c-cf0b204725cc None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:39.411738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89014274-0c1e-46a6-971c-cf0b204725cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:39.421544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8243b8eb-9b09-4895-82b1-fd3be54f58bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67a5cc6144c0434b9f16fb050c4c7662'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:39.422491 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8243b8eb-9b09-4895-82b1-fd3be54f58bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67a5cc6144c0434b9f16fb050c4c7662'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:39.423218 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8243b8eb-9b09-4895-82b1-fd3be54f58bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67a5cc6144c0434b9f16fb050c4c7662'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:39.441719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8243b8eb-9b09-4895-82b1-fd3be54f58bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67a5cc6144c0434b9f16fb050c4c7662'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:39.454426 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-8243b8eb-9b09-4895-82b1-fd3be54f58bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67a5cc6144c0434b9f16fb050c4c7662'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:39.462433 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-89014274-0c1e-46a6-971c-cf0b204725cc None None] MFA Rules not processed for user `166426445c654182a19d583b24242536`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:39.463752 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-89014274-0c1e-46a6-971c-cf0b204725cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:39.580978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 977/1949] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:39.584150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89014274-0c1e-46a6-971c-cf0b204725cc None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:39.584150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-89014274-0c1e-46a6-971c-cf0b204725cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:39.614283 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 973/1950] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:39 2022] DELETE /identity/v3/users/67a5cc6144c0434b9f16fb050c4c7662 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:39.644128 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:39.696026 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89014274-0c1e-46a6-971c-cf0b204725cc None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:39.783402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81499c4-80ab-4ca7-b475-74d6e34368c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:39.784794 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81499c4-80ab-4ca7-b475-74d6e34368c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:39.784794 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81499c4-80ab-4ca7-b475-74d6e34368c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:39.784794 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81499c4-80ab-4ca7-b475-74d6e34368c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:39.795773 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81499c4-80ab-4ca7-b475-74d6e34368c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:39.795773 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81499c4-80ab-4ca7-b475-74d6e34368c3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:39.937496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 978/1951] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:47:39 2022] POST /identity/v3/projects => generated 371 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:39.942206 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243b8eb-9b09-4895-82b1-fd3be54f58bf admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:39.953286 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243b8eb-9b09-4895-82b1-fd3be54f58bf admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:39.989792 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81499c4-80ab-4ca7-b475-74d6e34368c3 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:39.992153 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f81499c4-80ab-4ca7-b475-74d6e34368c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:40.049708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628f940-56d7-447c-8ca3-1a284d4facbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:40.050424 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628f940-56d7-447c-8ca3-1a284d4facbb admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:40.050706 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628f940-56d7-447c-8ca3-1a284d4facbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:40.051949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628f940-56d7-447c-8ca3-1a284d4facbb admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:40.051949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628f940-56d7-447c-8ca3-1a284d4facbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:40.055971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628f940-56d7-447c-8ca3-1a284d4facbb admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:40.088388 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81499c4-80ab-4ca7-b475-74d6e34368c3 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:40.094022 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1159e-d640-4b32-b191-81b2ebc4ace9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:40.094920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1159e-d640-4b32-b191-81b2ebc4ace9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:40.095204 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1159e-d640-4b32-b191-81b2ebc4ace9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:40.095488 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1159e-d640-4b32-b191-81b2ebc4ace9 admin admin] PATH_INFO: `/v3/projects/4a773b75a4b4442ca01dc2b0d9e4e5d9` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:40.096877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1159e-d640-4b32-b191-81b2ebc4ace9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a773b75a4b4442ca01dc2b0d9e4e5d9)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:40.098841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1159e-d640-4b32-b191-81b2ebc4ace9 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:40.180520 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 974/1952] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:47:39 2022] POST /identity/v3/users => generated 292 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:40.181945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628f940-56d7-447c-8ca3-1a284d4facbb admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:40.193719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628f940-56d7-447c-8ca3-1a284d4facbb admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:40.199466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64a432-4adb-4e1c-a75f-53cdf8810078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:40.201408 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64a432-4adb-4e1c-a75f-53cdf8810078 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:40.201707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64a432-4adb-4e1c-a75f-53cdf8810078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:40.202133 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64a432-4adb-4e1c-a75f-53cdf8810078 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:40.203283 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64a432-4adb-4e1c-a75f-53cdf8810078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:40.205320 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64a432-4adb-4e1c-a75f-53cdf8810078 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:40.221467 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 975/1953] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:40.223479 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64a432-4adb-4e1c-a75f-53cdf8810078 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:40.235627 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64a432-4adb-4e1c-a75f-53cdf8810078 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:40.236619 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-12e1159e-d640-4b32-b191-81b2ebc4ace9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a773b75a4b4442ca01dc2b0d9e4e5d9', 'request_id': 'req-12e1159e-d640-4b32-b191-81b2ebc4ace9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:40.244036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-12e1159e-d640-4b32-b191-81b2ebc4ace9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a773b75a4b4442ca01dc2b0d9e4e5d9', 'request_id': 'req-12e1159e-d640-4b32-b191-81b2ebc4ace9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:40.252669 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-12e1159e-d640-4b32-b191-81b2ebc4ace9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a773b75a4b4442ca01dc2b0d9e4e5d9', 'request_id': 'req-12e1159e-d640-4b32-b191-81b2ebc4ace9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:40.258301 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:40.269963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:40.309726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f817b0-dcf0-42ee-a32e-73be3ccbfa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:40.309726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f817b0-dcf0-42ee-a32e-73be3ccbfa3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:40.309726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f817b0-dcf0-42ee-a32e-73be3ccbfa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:40.309726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f817b0-dcf0-42ee-a32e-73be3ccbfa3e admin admin] PATH_INFO: `/v3/projects/6cb3085ae3394f1b96bcd4b09bb5deab/users/d5ad97d8ac274a74ac5b1f0f7f3ff578/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:40.323701 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f817b0-dcf0-42ee-a32e-73be3ccbfa3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cb3085ae3394f1b96bcd4b09bb5deab, user_id=d5ad97d8ac274a74ac5b1f0f7f3ff578, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:40.328296 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f817b0-dcf0-42ee-a32e-73be3ccbfa3e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:40.334461 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-12e1159e-d640-4b32-b191-81b2ebc4ace9 admin admin] The token cache is being invalidate because project 4a773b75a4b4442ca01dc2b0d9e4e5d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:40.334753 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-12e1159e-d640-4b32-b191-81b2ebc4ace9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:40.335441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-12e1159e-d640-4b32-b191-81b2ebc4ace9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:40.338451 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 979/1954] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:39 2022] DELETE /identity/v3/projects/4a773b75a4b4442ca01dc2b0d9e4e5d9 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:40.343042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f945939-0b96-4c5d-aa86-96e4ed3ccfa5 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:40.343334 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f945939-0b96-4c5d-aa86-96e4ed3ccfa5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:40.343613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f945939-0b96-4c5d-aa86-96e4ed3ccfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:40.362820 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-9f945939-0b96-4c5d-aa86-96e4ed3ccfa5 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:40.364184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9f945939-0b96-4c5d-aa86-96e4ed3ccfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:40.372970 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 980/1955] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:47:40.381748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7591f-9288-4707-9761-79b9be9be05f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:40.382036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7591f-9288-4707-9761-79b9be9be05f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:40.382313 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7591f-9288-4707-9761-79b9be9be05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:40.401226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-bde7591f-9288-4707-9761-79b9be9be05f None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:40.402422 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bde7591f-9288-4707-9761-79b9be9be05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:40.411116 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 981/1956] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:47:40.415235 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7591f-9288-4707-9761-79b9be9be05f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:40.424439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7591f-9288-4707-9761-79b9be9be05f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:40.430289 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb7adb-0fd5-4345-99e6-16618ad8856c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:40.430289 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb7adb-0fd5-4345-99e6-16618ad8856c admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:40.430289 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb7adb-0fd5-4345-99e6-16618ad8856c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:40.430289 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb7adb-0fd5-4345-99e6-16618ad8856c admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:40.431354 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb7adb-0fd5-4345-99e6-16618ad8856c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:47:40.431354 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb7adb-0fd5-4345-99e6-16618ad8856c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:40.432962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb7adb-0fd5-4345-99e6-16618ad8856c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:40.433766 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 976/1957] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:40 2022] PUT /identity/v3/projects/6cb3085ae3394f1b96bcd4b09bb5deab/users/d5ad97d8ac274a74ac5b1f0f7f3ff578/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:40.438347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f817b0-dcf0-42ee-a32e-73be3ccbfa3e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:40.443266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b6f817b0-dcf0-42ee-a32e-73be3ccbfa3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:40.451033 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 982/1958] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:47:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:47:40.452381 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb7adb-0fd5-4345-99e6-16618ad8856c admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:40.463700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb7adb-0fd5-4345-99e6-16618ad8856c admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:40.549572 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb59848-f17f-4cde-acbf-caef3a0fd260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:40.550724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb59848-f17f-4cde-acbf-caef3a0fd260 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:40.551238 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb59848-f17f-4cde-acbf-caef3a0fd260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:40.551782 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb59848-f17f-4cde-acbf-caef3a0fd260 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:40.552572 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f817b0-dcf0-42ee-a32e-73be3ccbfa3e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:40.553151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb59848-f17f-4cde-acbf-caef3a0fd260 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:40.562069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7adcb2b-f7c7-46ad-a62b-39deeb11125f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:40.563533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7adcb2b-f7c7-46ad-a62b-39deeb11125f admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:40.563533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7adcb2b-f7c7-46ad-a62b-39deeb11125f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:40.563785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7adcb2b-f7c7-46ad-a62b-39deeb11125f admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:40.565206 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7adcb2b-f7c7-46ad-a62b-39deeb11125f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:40.566666 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb59848-f17f-4cde-acbf-caef3a0fd260 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:40.568891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7adcb2b-f7c7-46ad-a62b-39deeb11125f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:40.589241 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 977/1959] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:40 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:40.594302 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7adcb2b-f7c7-46ad-a62b-39deeb11125f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:40.613998 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7adcb2b-f7c7-46ad-a62b-39deeb11125f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:40.623854 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c695c4-6540-4869-9921-9b5ed33a08ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:40.625038 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c695c4-6540-4869-9921-9b5ed33a08ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:40.625321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c695c4-6540-4869-9921-9b5ed33a08ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:40.625621 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c695c4-6540-4869-9921-9b5ed33a08ce admin admin] PATH_INFO: `/v3/projects/6cb3085ae3394f1b96bcd4b09bb5deab/users/d5ad97d8ac274a74ac5b1f0f7f3ff578/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:40.627896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c695c4-6540-4869-9921-9b5ed33a08ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cb3085ae3394f1b96bcd4b09bb5deab, user_id=d5ad97d8ac274a74ac5b1f0f7f3ff578, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:40.630628 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c695c4-6540-4869-9921-9b5ed33a08ce admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:40.645032 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 978/1960] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:40 2022] PUT /identity/v3/projects/6cb3085ae3394f1b96bcd4b09bb5deab/users/d5ad97d8ac274a74ac5b1f0f7f3ff578/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:40.656612 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e27a62-6c59-4973-ad68-6f887edcbf6b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:40.656910 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e27a62-6c59-4973-ad68-6f887edcbf6b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:40.657198 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e27a62-6c59-4973-ad68-6f887edcbf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:40.687960 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d2e27a62-6c59-4973-ad68-6f887edcbf6b None None] MFA Rules not processed for user `d5ad97d8ac274a74ac5b1f0f7f3ff578`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:40.689247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d2e27a62-6c59-4973-ad68-6f887edcbf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:40.707639 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 983/1961] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:47:40 2022] POST /identity/v3/projects => generated 371 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:40.713393 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb59848-f17f-4cde-acbf-caef3a0fd260 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:40.817572 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb59848-f17f-4cde-acbf-caef3a0fd260 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:40.820470 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 979/1962] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:40 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:47:40.830229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f87154-4c3d-4c84-93b1-44754cd644d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:40.831305 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f87154-4c3d-4c84-93b1-44754cd644d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:40.832001 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f87154-4c3d-4c84-93b1-44754cd644d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:40.832290 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f87154-4c3d-4c84-93b1-44754cd644d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:40.833306 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f87154-4c3d-4c84-93b1-44754cd644d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:40.835226 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f87154-4c3d-4c84-93b1-44754cd644d4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:40.875316 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e27a62-6c59-4973-ad68-6f887edcbf6b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:40.876954 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d2e27a62-6c59-4973-ad68-6f887edcbf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:40.958013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e27a62-6c59-4973-ad68-6f887edcbf6b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:40.963460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 984/1963] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:47:40 2022] POST /identity/v3/users => generated 286 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:40.969181 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f87154-4c3d-4c84-93b1-44754cd644d4 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:40.982355 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f87154-4c3d-4c84-93b1-44754cd644d4 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:40.987943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791437e2-30b7-41fa-a398-d4386af98d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:40.988639 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791437e2-30b7-41fa-a398-d4386af98d02 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:40.988914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791437e2-30b7-41fa-a398-d4386af98d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:40.989193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791437e2-30b7-41fa-a398-d4386af98d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:40.989978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791437e2-30b7-41fa-a398-d4386af98d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:40.991612 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791437e2-30b7-41fa-a398-d4386af98d02 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:41.009316 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 985/1964] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:41.010410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791437e2-30b7-41fa-a398-d4386af98d02 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:41.022129 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7bff74-682b-4948-8445-e4e4784ce236 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:41.022568 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791437e2-30b7-41fa-a398-d4386af98d02 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:41.023396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bff74-682b-4948-8445-e4e4784ce236 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:41.023890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bff74-682b-4948-8445-e4e4784ce236 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:41.024358 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bff74-682b-4948-8445-e4e4784ce236 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:41.027797 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce25cce-2048-423e-bfd6-106c00d654ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:41.028964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce25cce-2048-423e-bfd6-106c00d654ec admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:41.029440 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce25cce-2048-423e-bfd6-106c00d654ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:41.029947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce25cce-2048-423e-bfd6-106c00d654ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:41.031154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce25cce-2048-423e-bfd6-106c00d654ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:41.033202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce25cce-2048-423e-bfd6-106c00d654ec admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:41.040810 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:41.047347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 986/1965] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:41.051817 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce25cce-2048-423e-bfd6-106c00d654ec admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:41.064430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce25cce-2048-423e-bfd6-106c00d654ec admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:41.074323 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bc832-c72c-4bdb-97e6-64f64bbd3bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:41.074323 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bc832-c72c-4bdb-97e6-64f64bbd3bda admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:41.074323 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bc832-c72c-4bdb-97e6-64f64bbd3bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:41.074323 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bc832-c72c-4bdb-97e6-64f64bbd3bda admin admin] PATH_INFO: `/v3/projects/279a903930f443a393c257c495ee0cca/users/73c59ce0eb124e49b66ee3b63bafe764/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:41.094649 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62bc832-c72c-4bdb-97e6-64f64bbd3bda admin admin] RBAC: Authorizing `identity:create_grant(project_id=279a903930f443a393c257c495ee0cca, user_id=73c59ce0eb124e49b66ee3b63bafe764, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:41.100084 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62bc832-c72c-4bdb-97e6-64f64bbd3bda admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:41.121470 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7bff74-682b-4948-8445-e4e4784ce236 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:41.124094 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7bff74-682b-4948-8445-e4e4784ce236 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:41.136278 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 980/1966] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:40 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:47:41.193535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:41.199217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 987/1967] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:41 2022] PUT /identity/v3/projects/279a903930f443a393c257c495ee0cca/users/73c59ce0eb124e49b66ee3b63bafe764/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:41.217568 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c3193-aa83-4197-9732-dd09eb582e7c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:41.218484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c3193-aa83-4197-9732-dd09eb582e7c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:41.218484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c3193-aa83-4197-9732-dd09eb582e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:41.254297 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-133c3193-aa83-4197-9732-dd09eb582e7c None None] MFA Rules not processed for user `73c59ce0eb124e49b66ee3b63bafe764`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:41.256892 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-133c3193-aa83-4197-9732-dd09eb582e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:41.339290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 981/1968] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:41.350719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bc832-c72c-4bdb-97e6-64f64bbd3bda admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:41.352461 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e62bc832-c72c-4bdb-97e6-64f64bbd3bda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:41.450737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bc832-c72c-4bdb-97e6-64f64bbd3bda admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:41.469220 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3be662-90b4-4afd-91eb-2cff19d62954 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:41.469986 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3be662-90b4-4afd-91eb-2cff19d62954 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:41.475839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3be662-90b4-4afd-91eb-2cff19d62954 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:41.475839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3be662-90b4-4afd-91eb-2cff19d62954 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:41.480605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3be662-90b4-4afd-91eb-2cff19d62954 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:41.483180 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3be662-90b4-4afd-91eb-2cff19d62954 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:41.493613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 988/1969] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:47:42.643300 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c3193-aa83-4197-9732-dd09eb582e7c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:42.656628 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:42.712615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c3193-aa83-4197-9732-dd09eb582e7c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:42.718159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc9b99-7bb5-4d07-929f-50d8c8c3d0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:42.719038 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc9b99-7bb5-4d07-929f-50d8c8c3d0a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:42.721678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc9b99-7bb5-4d07-929f-50d8c8c3d0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:42.721678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc9b99-7bb5-4d07-929f-50d8c8c3d0a3 admin admin] PATH_INFO: `/v3/users/166426445c654182a19d583b24242536` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:42.721678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc9b99-7bb5-4d07-929f-50d8c8c3d0a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=166426445c654182a19d583b24242536)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:42.722844 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc9b99-7bb5-4d07-929f-50d8c8c3d0a3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:42.885910 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-68dc9b99-7bb5-4d07-929f-50d8c8c3d0a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '166426445c654182a19d583b24242536'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:42.886708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-68dc9b99-7bb5-4d07-929f-50d8c8c3d0a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '166426445c654182a19d583b24242536'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:42.887430 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-68dc9b99-7bb5-4d07-929f-50d8c8c3d0a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '166426445c654182a19d583b24242536'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:42.901786 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-68dc9b99-7bb5-4d07-929f-50d8c8c3d0a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '166426445c654182a19d583b24242536'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:42.910634 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-68dc9b99-7bb5-4d07-929f-50d8c8c3d0a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '166426445c654182a19d583b24242536'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:42.998743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 982/1970] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:42 2022] DELETE /identity/v3/users/166426445c654182a19d583b24242536 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:43.003661 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf76d2-7a78-4cbe-bdb1-a97c98f7d00e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:43.004162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf76d2-7a78-4cbe-bdb1-a97c98f7d00e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:43.004274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf76d2-7a78-4cbe-bdb1-a97c98f7d00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:43.023125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-1daf76d2-7a78-4cbe-bdb1-a97c98f7d00e None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:43.024366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1daf76d2-7a78-4cbe-bdb1-a97c98f7d00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:43.101959 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 989/1971] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:43.114004 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc9b99-7bb5-4d07-929f-50d8c8c3d0a3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:43.116335 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-68dc9b99-7bb5-4d07-929f-50d8c8c3d0a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:43.195444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc9b99-7bb5-4d07-929f-50d8c8c3d0a3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:43.199211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a657d4-2309-45cb-9c65-5152f17ad295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:43.200454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a657d4-2309-45cb-9c65-5152f17ad295 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:43.200838 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a657d4-2309-45cb-9c65-5152f17ad295 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:43.201344 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a657d4-2309-45cb-9c65-5152f17ad295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:43.211606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a657d4-2309-45cb-9c65-5152f17ad295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:43.213117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a657d4-2309-45cb-9c65-5152f17ad295 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:43.224436 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 983/1972] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:47:43.699221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf76d2-7a78-4cbe-bdb1-a97c98f7d00e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:43.701829 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1daf76d2-7a78-4cbe-bdb1-a97c98f7d00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:43.728719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf76d2-7a78-4cbe-bdb1-a97c98f7d00e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:43.734885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa5de5-632a-4483-949b-ae84bd8df534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:43.735867 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa5de5-632a-4483-949b-ae84bd8df534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:43.736155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa5de5-632a-4483-949b-ae84bd8df534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:43.736442 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa5de5-632a-4483-949b-ae84bd8df534 admin admin] PATH_INFO: `/v3/projects/2583fb3db2dc45a297d10d0e71db6f51` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:43.737849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa5de5-632a-4483-949b-ae84bd8df534 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2583fb3db2dc45a297d10d0e71db6f51)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:43.741535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa5de5-632a-4483-949b-ae84bd8df534 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:43.858196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-46fa5de5-632a-4483-949b-ae84bd8df534 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2583fb3db2dc45a297d10d0e71db6f51', 'request_id': 'req-46fa5de5-632a-4483-949b-ae84bd8df534'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:43.864162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-46fa5de5-632a-4483-949b-ae84bd8df534 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2583fb3db2dc45a297d10d0e71db6f51', 'request_id': 'req-46fa5de5-632a-4483-949b-ae84bd8df534'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:43.871053 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-46fa5de5-632a-4483-949b-ae84bd8df534 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2583fb3db2dc45a297d10d0e71db6f51', 'request_id': 'req-46fa5de5-632a-4483-949b-ae84bd8df534'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:43.953415 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-46fa5de5-632a-4483-949b-ae84bd8df534 admin admin] The token cache is being invalidate because project 2583fb3db2dc45a297d10d0e71db6f51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:43.953700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-46fa5de5-632a-4483-949b-ae84bd8df534 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:43.954361 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-46fa5de5-632a-4483-949b-ae84bd8df534 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:43.955971 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 990/1973] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:43 2022] DELETE /identity/v3/projects/2583fb3db2dc45a297d10d0e71db6f51 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:43.960284 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a657d4-2309-45cb-9c65-5152f17ad295 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:43.971045 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a657d4-2309-45cb-9c65-5152f17ad295 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:44.002395 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:44.033350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6289ef-5174-4f34-b529-48c80955ba51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:44.034240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6289ef-5174-4f34-b529-48c80955ba51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:44.034524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6289ef-5174-4f34-b529-48c80955ba51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:44.034793 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6289ef-5174-4f34-b529-48c80955ba51 admin admin] PATH_INFO: `/v3/users/d5ad97d8ac274a74ac5b1f0f7f3ff578` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:44.036138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6289ef-5174-4f34-b529-48c80955ba51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5ad97d8ac274a74ac5b1f0f7f3ff578)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:44.038137 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6289ef-5174-4f34-b529-48c80955ba51 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:44.233211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dd6289ef-5174-4f34-b529-48c80955ba51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5ad97d8ac274a74ac5b1f0f7f3ff578'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:44.233967 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dd6289ef-5174-4f34-b529-48c80955ba51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5ad97d8ac274a74ac5b1f0f7f3ff578'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:44.234587 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dd6289ef-5174-4f34-b529-48c80955ba51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5ad97d8ac274a74ac5b1f0f7f3ff578'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:44.248636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dd6289ef-5174-4f34-b529-48c80955ba51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5ad97d8ac274a74ac5b1f0f7f3ff578'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:44.256072 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dd6289ef-5174-4f34-b529-48c80955ba51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5ad97d8ac274a74ac5b1f0f7f3ff578'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:44.416134 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 984/1974] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:43 2022] DELETE /identity/v3/users/d5ad97d8ac274a74ac5b1f0f7f3ff578 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:44.971137 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa5de5-632a-4483-949b-ae84bd8df534 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:44.972819 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-46fa5de5-632a-4483-949b-ae84bd8df534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:44.987572 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa5de5-632a-4483-949b-ae84bd8df534 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:44.992140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:45.050196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5df8c-8a72-4839-bb9b-de53580b5d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:45.051063 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5df8c-8a72-4839-bb9b-de53580b5d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:45.051348 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5df8c-8a72-4839-bb9b-de53580b5d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:45.051629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5df8c-8a72-4839-bb9b-de53580b5d32 admin admin] PATH_INFO: `/v3/projects/6cb3085ae3394f1b96bcd4b09bb5deab` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:45.053371 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5df8c-8a72-4839-bb9b-de53580b5d32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cb3085ae3394f1b96bcd4b09bb5deab)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:45.054993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5df8c-8a72-4839-bb9b-de53580b5d32 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:45.185754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-44c5df8c-8a72-4839-bb9b-de53580b5d32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cb3085ae3394f1b96bcd4b09bb5deab', 'request_id': 'req-44c5df8c-8a72-4839-bb9b-de53580b5d32'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:45.192819 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-44c5df8c-8a72-4839-bb9b-de53580b5d32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cb3085ae3394f1b96bcd4b09bb5deab', 'request_id': 'req-44c5df8c-8a72-4839-bb9b-de53580b5d32'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:45.200643 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-44c5df8c-8a72-4839-bb9b-de53580b5d32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cb3085ae3394f1b96bcd4b09bb5deab', 'request_id': 'req-44c5df8c-8a72-4839-bb9b-de53580b5d32'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:45.301307 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-44c5df8c-8a72-4839-bb9b-de53580b5d32 admin admin] The token cache is being invalidate because project 6cb3085ae3394f1b96bcd4b09bb5deab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:45.301521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-44c5df8c-8a72-4839-bb9b-de53580b5d32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:45.302387 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-44c5df8c-8a72-4839-bb9b-de53580b5d32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:45.304349 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 991/1975] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:44 2022] DELETE /identity/v3/projects/6cb3085ae3394f1b96bcd4b09bb5deab => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:45.310805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809dad3-d16d-48b2-bf70-43d016684771 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:45.310805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809dad3-d16d-48b2-bf70-43d016684771 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:45.310805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809dad3-d16d-48b2-bf70-43d016684771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:45.329923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f809dad3-d16d-48b2-bf70-43d016684771 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:45.331131 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f809dad3-d16d-48b2-bf70-43d016684771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:45.415141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 985/1976] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:45.423607 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64783d6e-24e9-41d2-b2c1-6c00f1928447 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:45.424047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64783d6e-24e9-41d2-b2c1-6c00f1928447 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:45.424335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64783d6e-24e9-41d2-b2c1-6c00f1928447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:45.444203 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-64783d6e-24e9-41d2-b2c1-6c00f1928447 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:45.445442 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-64783d6e-24e9-41d2-b2c1-6c00f1928447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:45.455242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 992/1977] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:45.459836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809dad3-d16d-48b2-bf70-43d016684771 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:45.471540 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809dad3-d16d-48b2-bf70-43d016684771 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:45.476723 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f4a2b-7391-4cdb-9e98-9a859f40b99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:45.477400 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f4a2b-7391-4cdb-9e98-9a859f40b99c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:45.477733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f4a2b-7391-4cdb-9e98-9a859f40b99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:45.478015 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f4a2b-7391-4cdb-9e98-9a859f40b99c admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:45.478758 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f4a2b-7391-4cdb-9e98-9a859f40b99c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:47:45.479076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f4a2b-7391-4cdb-9e98-9a859f40b99c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:45.480813 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f4a2b-7391-4cdb-9e98-9a859f40b99c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:45.506785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 986/1978] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:47:45.506949 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64783d6e-24e9-41d2-b2c1-6c00f1928447 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:45.518185 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64783d6e-24e9-41d2-b2c1-6c00f1928447 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:45.525783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61440032-9d6c-4c67-9eda-8c41bd8f0dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:45.526479 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61440032-9d6c-4c67-9eda-8c41bd8f0dfa admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:45.529619 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61440032-9d6c-4c67-9eda-8c41bd8f0dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:45.529619 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61440032-9d6c-4c67-9eda-8c41bd8f0dfa admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:45.529619 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61440032-9d6c-4c67-9eda-8c41bd8f0dfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:45.530046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61440032-9d6c-4c67-9eda-8c41bd8f0dfa admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:45.623721 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 993/1979] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:47:45 2022] POST /identity/v3/projects => generated 381 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:45.627909 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f4a2b-7391-4cdb-9e98-9a859f40b99c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:45.646477 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f4a2b-7391-4cdb-9e98-9a859f40b99c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:45.696206 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:45.720031 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716c7a8-8d5f-455e-9427-1db8229b3ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:45.720725 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716c7a8-8d5f-455e-9427-1db8229b3ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:45.720994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716c7a8-8d5f-455e-9427-1db8229b3ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:45.721273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716c7a8-8d5f-455e-9427-1db8229b3ae2 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:45.746467 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5716c7a8-8d5f-455e-9427-1db8229b3ae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:45.746467 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5716c7a8-8d5f-455e-9427-1db8229b3ae2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:45.852152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 987/1980] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:47:45 2022] POST /identity/v3/users => generated 291 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:45.866907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61440032-9d6c-4c67-9eda-8c41bd8f0dfa admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:45.874170 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61440032-9d6c-4c67-9eda-8c41bd8f0dfa admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:45.881828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966d5b1-abb0-43be-9af1-12d1abcd1723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:45.882999 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966d5b1-abb0-43be-9af1-12d1abcd1723 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:45.883374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966d5b1-abb0-43be-9af1-12d1abcd1723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:45.883877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966d5b1-abb0-43be-9af1-12d1abcd1723 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:45.885219 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966d5b1-abb0-43be-9af1-12d1abcd1723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:45.888279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966d5b1-abb0-43be-9af1-12d1abcd1723 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:45.908466 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 994/1981] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:45.915035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716c7a8-8d5f-455e-9427-1db8229b3ae2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:45.941626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716c7a8-8d5f-455e-9427-1db8229b3ae2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:45.952490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377c081-4df0-43e0-b079-9d4867e6f446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:45.955953 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377c081-4df0-43e0-b079-9d4867e6f446 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:45.956939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377c081-4df0-43e0-b079-9d4867e6f446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:45.963981 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377c081-4df0-43e0-b079-9d4867e6f446 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:45.963981 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377c081-4df0-43e0-b079-9d4867e6f446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:45.963981 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377c081-4df0-43e0-b079-9d4867e6f446 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:45.978236 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 988/1982] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:45.986191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966d5b1-abb0-43be-9af1-12d1abcd1723 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:45.998071 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966d5b1-abb0-43be-9af1-12d1abcd1723 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:46.003077 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ebd0b-a24c-4e8c-9179-bbbf76fd7a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:46.004046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ebd0b-a24c-4e8c-9179-bbbf76fd7a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:46.004328 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ebd0b-a24c-4e8c-9179-bbbf76fd7a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:46.004604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ebd0b-a24c-4e8c-9179-bbbf76fd7a2b admin admin] PATH_INFO: `/v3/projects/d3ce0c562bf145939da6703b00c2b99b/users/c52e77d8ab0f47e9a60113193a8872e8/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:46.023898 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086ebd0b-a24c-4e8c-9179-bbbf76fd7a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3ce0c562bf145939da6703b00c2b99b, user_id=c52e77d8ab0f47e9a60113193a8872e8, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:46.026599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086ebd0b-a24c-4e8c-9179-bbbf76fd7a2b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:46.131265 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 995/1983] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:45 2022] PUT /identity/v3/projects/d3ce0c562bf145939da6703b00c2b99b/users/c52e77d8ab0f47e9a60113193a8872e8/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:46.144556 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967666b-c266-42c9-9857-cd5289cf341f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:46.145055 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967666b-c266-42c9-9857-cd5289cf341f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:46.145585 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967666b-c266-42c9-9857-cd5289cf341f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:46.198936 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e967666b-c266-42c9-9857-cd5289cf341f None None] MFA Rules not processed for user `c52e77d8ab0f47e9a60113193a8872e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:46.201693 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e967666b-c266-42c9-9857-cd5289cf341f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:46.329068 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 989/1984] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:46.335020 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ebd0b-a24c-4e8c-9179-bbbf76fd7a2b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:46.377766 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:46.416265 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ebd0b-a24c-4e8c-9179-bbbf76fd7a2b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:46.421929 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613350c-42bc-4fec-a35e-b0904ec4508e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:46.422608 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613350c-42bc-4fec-a35e-b0904ec4508e admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:46.422891 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613350c-42bc-4fec-a35e-b0904ec4508e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:46.423174 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613350c-42bc-4fec-a35e-b0904ec4508e admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:46.423922 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613350c-42bc-4fec-a35e-b0904ec4508e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:46.425970 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613350c-42bc-4fec-a35e-b0904ec4508e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:46.525809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 996/1985] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:47:46 2022] POST /identity/v3/projects => generated 379 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:47:46.534446 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967666b-c266-42c9-9857-cd5289cf341f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:46.546587 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967666b-c266-42c9-9857-cd5289cf341f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:46.651349 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2380608-55cb-4a4d-9ae1-9d4250cfc1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:46.652291 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2380608-55cb-4a4d-9ae1-9d4250cfc1b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:46.652470 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2380608-55cb-4a4d-9ae1-9d4250cfc1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:46.652671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2380608-55cb-4a4d-9ae1-9d4250cfc1b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:46.654584 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2380608-55cb-4a4d-9ae1-9d4250cfc1b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:46.656674 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2380608-55cb-4a4d-9ae1-9d4250cfc1b6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:46.750107 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 990/1986] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:47:46 2022] POST /identity/v3/users => generated 296 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:47:46.754654 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613350c-42bc-4fec-a35e-b0904ec4508e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:46.766413 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613350c-42bc-4fec-a35e-b0904ec4508e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:46.772795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797819e-ff68-4439-ae59-e0f7a9a68f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:46.773920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797819e-ff68-4439-ae59-e0f7a9a68f4f admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:46.774527 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797819e-ff68-4439-ae59-e0f7a9a68f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:46.774898 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797819e-ff68-4439-ae59-e0f7a9a68f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:46.775760 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f797819e-ff68-4439-ae59-e0f7a9a68f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:46.779074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f797819e-ff68-4439-ae59-e0f7a9a68f4f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:46.796951 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 997/1987] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:47:46.801472 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2380608-55cb-4a4d-9ae1-9d4250cfc1b6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:46.824270 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2380608-55cb-4a4d-9ae1-9d4250cfc1b6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:46.832676 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb4685-9653-462e-a6e3-4309ff759c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:46.833747 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb4685-9653-462e-a6e3-4309ff759c7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:46.839447 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb4685-9653-462e-a6e3-4309ff759c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:46.839833 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb4685-9653-462e-a6e3-4309ff759c7a admin admin] PATH_INFO: `/v3/projects/e66ea819617644b18b43ee4565c28b4e/users/51edb1a774ed45d1ba1422fb456ad927/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:46.874934 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb4685-9653-462e-a6e3-4309ff759c7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e66ea819617644b18b43ee4565c28b4e, user_id=51edb1a774ed45d1ba1422fb456ad927, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:46.880429 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb4685-9653-462e-a6e3-4309ff759c7a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:47.022936 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 991/1988] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:46 2022] PUT /identity/v3/projects/e66ea819617644b18b43ee4565c28b4e/users/51edb1a774ed45d1ba1422fb456ad927/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:47.027356 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797819e-ff68-4439-ae59-e0f7a9a68f4f admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:47.039543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797819e-ff68-4439-ae59-e0f7a9a68f4f admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:47.123976 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27850d0f-bda4-4914-822b-abc0a8d3e97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:47.124587 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27850d0f-bda4-4914-822b-abc0a8d3e97d admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:47.125074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27850d0f-bda4-4914-822b-abc0a8d3e97d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:47.125362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27850d0f-bda4-4914-822b-abc0a8d3e97d admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:47.128778 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27850d0f-bda4-4914-822b-abc0a8d3e97d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:47.134886 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27850d0f-bda4-4914-822b-abc0a8d3e97d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:47.146961 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 998/1989] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:47.151382 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb4685-9653-462e-a6e3-4309ff759c7a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:47.162480 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb4685-9653-462e-a6e3-4309ff759c7a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:47.167607 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9ec7e-7c42-4e21-a983-46fb87d70e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:47.169070 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9ec7e-7c42-4e21-a983-46fb87d70e0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:47.169350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9ec7e-7c42-4e21-a983-46fb87d70e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:47.169626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9ec7e-7c42-4e21-a983-46fb87d70e0c admin admin] PATH_INFO: `/v3/projects/e66ea819617644b18b43ee4565c28b4e/users/51edb1a774ed45d1ba1422fb456ad927/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:47.171983 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9ec7e-7c42-4e21-a983-46fb87d70e0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e66ea819617644b18b43ee4565c28b4e, user_id=51edb1a774ed45d1ba1422fb456ad927, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:47.174725 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9ec7e-7c42-4e21-a983-46fb87d70e0c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:47.182818 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:47.188672 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 992/1990] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:47 2022] PUT /identity/v3/projects/e66ea819617644b18b43ee4565c28b4e/users/51edb1a774ed45d1ba1422fb456ad927/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:47.202251 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d58fda-bdd5-4e7b-b5d4-6e8cb2ceb5e0 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:47.202942 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d58fda-bdd5-4e7b-b5d4-6e8cb2ceb5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:47.203297 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d58fda-bdd5-4e7b-b5d4-6e8cb2ceb5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:47.234698 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d0d58fda-bdd5-4e7b-b5d4-6e8cb2ceb5e0 None None] MFA Rules not processed for user `51edb1a774ed45d1ba1422fb456ad927`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:47.236035 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d0d58fda-bdd5-4e7b-b5d4-6e8cb2ceb5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:47.318534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 999/1991] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:47 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:47.334814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9ec7e-7c42-4e21-a983-46fb87d70e0c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:47.336773 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-62f9ec7e-7c42-4e21-a983-46fb87d70e0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:47.401230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9ec7e-7c42-4e21-a983-46fb87d70e0c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:47.405613 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808aad51-c1b4-4650-8cc8-1abf8df7094b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:47.406282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808aad51-c1b4-4650-8cc8-1abf8df7094b service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:47.406567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808aad51-c1b4-4650-8cc8-1abf8df7094b service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:47.406846 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808aad51-c1b4-4650-8cc8-1abf8df7094b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:47.417055 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808aad51-c1b4-4650-8cc8-1abf8df7094b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:47.418462 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808aad51-c1b4-4650-8cc8-1abf8df7094b service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:47.429548 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 993/1992] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:47:47.540688 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d58fda-bdd5-4e7b-b5d4-6e8cb2ceb5e0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:47.540688 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d0d58fda-bdd5-4e7b-b5d4-6e8cb2ceb5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:47.589954 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:47.636946 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d58fda-bdd5-4e7b-b5d4-6e8cb2ceb5e0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:47.642679 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb762e-89f5-4cef-9fac-07347749c77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:47.643761 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb762e-89f5-4cef-9fac-07347749c77e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:47.644046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb762e-89f5-4cef-9fac-07347749c77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:47.644327 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb762e-89f5-4cef-9fac-07347749c77e admin admin] PATH_INFO: `/v3/users/73c59ce0eb124e49b66ee3b63bafe764` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:47.645555 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb762e-89f5-4cef-9fac-07347749c77e admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c59ce0eb124e49b66ee3b63bafe764)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:47.647607 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb762e-89f5-4cef-9fac-07347749c77e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:47.867267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-bbcb762e-89f5-4cef-9fac-07347749c77e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c59ce0eb124e49b66ee3b63bafe764'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:47.868021 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-bbcb762e-89f5-4cef-9fac-07347749c77e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c59ce0eb124e49b66ee3b63bafe764'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:47.868686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-bbcb762e-89f5-4cef-9fac-07347749c77e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c59ce0eb124e49b66ee3b63bafe764'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:47.891152 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-bbcb762e-89f5-4cef-9fac-07347749c77e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c59ce0eb124e49b66ee3b63bafe764'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:47.905291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-bbcb762e-89f5-4cef-9fac-07347749c77e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c59ce0eb124e49b66ee3b63bafe764'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:47.994164 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1000/1993] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:47 2022] DELETE /identity/v3/users/73c59ce0eb124e49b66ee3b63bafe764 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:48.001238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e68bf-773e-4e76-aaf7-c7475894a761 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:48.001697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e68bf-773e-4e76-aaf7-c7475894a761 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:48.002058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e68bf-773e-4e76-aaf7-c7475894a761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:48.024574 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a16e68bf-773e-4e76-aaf7-c7475894a761 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:48.026621 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a16e68bf-773e-4e76-aaf7-c7475894a761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:48.119260 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 994/1994] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:48.131894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb762e-89f5-4cef-9fac-07347749c77e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:48.134261 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bbcb762e-89f5-4cef-9fac-07347749c77e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:48.222582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb762e-89f5-4cef-9fac-07347749c77e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:48.226705 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96058f-35b0-4cf6-8f93-fd50f18b5f55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:48.227506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96058f-35b0-4cf6-8f93-fd50f18b5f55 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:48.227970 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96058f-35b0-4cf6-8f93-fd50f18b5f55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:48.228293 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96058f-35b0-4cf6-8f93-fd50f18b5f55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:48.239250 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96058f-35b0-4cf6-8f93-fd50f18b5f55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:48.240771 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96058f-35b0-4cf6-8f93-fd50f18b5f55 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:48.252554 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1001/1995] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:47:48.829573 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e68bf-773e-4e76-aaf7-c7475894a761 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:48.831326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a16e68bf-773e-4e76-aaf7-c7475894a761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:48.849529 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e68bf-773e-4e76-aaf7-c7475894a761 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:48.857644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f91438-127c-4a10-ac0d-3943f8cb7ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:48.859173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f91438-127c-4a10-ac0d-3943f8cb7ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:48.859659 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f91438-127c-4a10-ac0d-3943f8cb7ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:48.863971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f91438-127c-4a10-ac0d-3943f8cb7ec9 admin admin] PATH_INFO: `/v3/projects/279a903930f443a393c257c495ee0cca` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:48.863971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f91438-127c-4a10-ac0d-3943f8cb7ec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=279a903930f443a393c257c495ee0cca)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:48.865794 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f91438-127c-4a10-ac0d-3943f8cb7ec9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:48.989459 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96058f-35b0-4cf6-8f93-fd50f18b5f55 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:48.989459 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1c96058f-35b0-4cf6-8f93-fd50f18b5f55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:49.043066 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:49.049147 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-32f91438-127c-4a10-ac0d-3943f8cb7ec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '279a903930f443a393c257c495ee0cca', 'request_id': 'req-32f91438-127c-4a10-ac0d-3943f8cb7ec9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:49.054887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-32f91438-127c-4a10-ac0d-3943f8cb7ec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '279a903930f443a393c257c495ee0cca', 'request_id': 'req-32f91438-127c-4a10-ac0d-3943f8cb7ec9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:49.057051 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96058f-35b0-4cf6-8f93-fd50f18b5f55 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:49.061827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-32f91438-127c-4a10-ac0d-3943f8cb7ec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '279a903930f443a393c257c495ee0cca', 'request_id': 'req-32f91438-127c-4a10-ac0d-3943f8cb7ec9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:49.125106 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24967279-cb54-4f86-b1cf-1ad50d358490 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51edb1a774ed45d1ba1422fb456ad927', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e66ea819617644b18b43ee4565c28b4e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:49.125786 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24967279-cb54-4f86-b1cf-1ad50d358490 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:49.126077 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24967279-cb54-4f86-b1cf-1ad50d358490 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:49.126358 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24967279-cb54-4f86-b1cf-1ad50d358490 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:49.127261 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24967279-cb54-4f86-b1cf-1ad50d358490 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:47:49.127578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24967279-cb54-4f86-b1cf-1ad50d358490 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:49.129221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24967279-cb54-4f86-b1cf-1ad50d358490 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:49.143252 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1002/1996] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:47:49.147464 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24967279-cb54-4f86-b1cf-1ad50d358490 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:49.158596 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24967279-cb54-4f86-b1cf-1ad50d358490 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:49.163342 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328649b-3d0a-4954-9fb9-f2f10d5b7038 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51edb1a774ed45d1ba1422fb456ad927', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e66ea819617644b18b43ee4565c28b4e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:49.164072 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328649b-3d0a-4954-9fb9-f2f10d5b7038 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:49.164360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328649b-3d0a-4954-9fb9-f2f10d5b7038 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:49.164647 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328649b-3d0a-4954-9fb9-f2f10d5b7038 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:49.165346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328649b-3d0a-4954-9fb9-f2f10d5b7038 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:49.167380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328649b-3d0a-4954-9fb9-f2f10d5b7038 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:49.238388 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-32f91438-127c-4a10-ac0d-3943f8cb7ec9 admin admin] The token cache is being invalidate because project 279a903930f443a393c257c495ee0cca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:49.247296 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-32f91438-127c-4a10-ac0d-3943f8cb7ec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:49.247296 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-32f91438-127c-4a10-ac0d-3943f8cb7ec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:49.247296 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 995/1997] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:48 2022] DELETE /identity/v3/projects/279a903930f443a393c257c495ee0cca => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:49.247610 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27610e9e-e9af-4988-99bf-efa3fe5151b5 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:49.247840 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27610e9e-e9af-4988-99bf-efa3fe5151b5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:49.248012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27610e9e-e9af-4988-99bf-efa3fe5151b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:49.270131 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-27610e9e-e9af-4988-99bf-efa3fe5151b5 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:49.271758 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-27610e9e-e9af-4988-99bf-efa3fe5151b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:49.294825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:49.338355 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 996/1998] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:47:49.346887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c553d-da02-4a6e-a002-f7af8e458d99 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:49.347182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c553d-da02-4a6e-a002-f7af8e458d99 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:49.347466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c553d-da02-4a6e-a002-f7af8e458d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:49.353118 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1003/1999] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:47:49 2022] POST /identity/v3/projects => generated 354 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:49.354405 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328649b-3d0a-4954-9fb9-f2f10d5b7038 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:49.355847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e328649b-3d0a-4954-9fb9-f2f10d5b7038 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:49.368580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c80c553d-da02-4a6e-a002-f7af8e458d99 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:49.369978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c80c553d-da02-4a6e-a002-f7af8e458d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:49.434718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328649b-3d0a-4954-9fb9-f2f10d5b7038 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:49.436945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 997/2000] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:49.440146 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720df865-d5ea-4fe5-95b6-c4cf95c1cb6e tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51edb1a774ed45d1ba1422fb456ad927', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e66ea819617644b18b43ee4565c28b4e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:49.440827 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720df865-d5ea-4fe5-95b6-c4cf95c1cb6e tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:49.441080 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c553d-da02-4a6e-a002-f7af8e458d99 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:49.441189 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720df865-d5ea-4fe5-95b6-c4cf95c1cb6e tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:49.441408 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720df865-d5ea-4fe5-95b6-c4cf95c1cb6e tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:49.442146 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720df865-d5ea-4fe5-95b6-c4cf95c1cb6e tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:47:49.442456 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720df865-d5ea-4fe5-95b6-c4cf95c1cb6e tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:49.445359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720df865-d5ea-4fe5-95b6-c4cf95c1cb6e tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:49.451560 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c553d-da02-4a6e-a002-f7af8e458d99 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:49.460447 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1004/2001] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:47:49.470942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb22ca-ccaf-4ba8-9736-bbcc8ade93f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:49.471945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb22ca-ccaf-4ba8-9736-bbcc8ade93f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:49.471945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb22ca-ccaf-4ba8-9736-bbcc8ade93f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:49.472509 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb22ca-ccaf-4ba8-9736-bbcc8ade93f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:49.473805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb22ca-ccaf-4ba8-9736-bbcc8ade93f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:47:49.473805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb22ca-ccaf-4ba8-9736-bbcc8ade93f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:49.475316 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb22ca-ccaf-4ba8-9736-bbcc8ade93f1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:49.490003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 998/2002] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:47:49.495182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720df865-d5ea-4fe5-95b6-c4cf95c1cb6e tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:49.504866 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb22ca-ccaf-4ba8-9736-bbcc8ade93f1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:49.509101 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720df865-d5ea-4fe5-95b6-c4cf95c1cb6e tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:49.515016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b6fba-479a-40da-9758-58d6327f8439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:49.515907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b6fba-479a-40da-9758-58d6327f8439 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:49.516367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b6fba-479a-40da-9758-58d6327f8439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:49.516653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b6fba-479a-40da-9758-58d6327f8439 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:49.516783 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb22ca-ccaf-4ba8-9736-bbcc8ade93f1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:49.517512 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b6fba-479a-40da-9758-58d6327f8439 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:49.519516 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b6fba-479a-40da-9758-58d6327f8439 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:49.524764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57801729-10f1-4179-901c-f37f636d3569 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51edb1a774ed45d1ba1422fb456ad927', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e66ea819617644b18b43ee4565c28b4e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:49.525870 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57801729-10f1-4179-901c-f37f636d3569 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:49.526257 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57801729-10f1-4179-901c-f37f636d3569 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:49.526781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57801729-10f1-4179-901c-f37f636d3569 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] PATH_INFO: `/v3/projects/cb8d51663a8a41fe9b58e17f94f52d60` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:49.528968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57801729-10f1-4179-901c-f37f636d3569 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cb8d51663a8a41fe9b58e17f94f52d60)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:49.532164 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57801729-10f1-4179-901c-f37f636d3569 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:49.612691 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1005/2003] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:47:49 2022] POST /identity/v3/projects => generated 401 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:49.618492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b6fba-479a-40da-9758-58d6327f8439 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:49.662070 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-57801729-10f1-4179-901c-f37f636d3569 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb8d51663a8a41fe9b58e17f94f52d60', 'request_id': 'req-57801729-10f1-4179-901c-f37f636d3569'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:49.670208 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-57801729-10f1-4179-901c-f37f636d3569 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb8d51663a8a41fe9b58e17f94f52d60', 'request_id': 'req-57801729-10f1-4179-901c-f37f636d3569'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:49.681368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-57801729-10f1-4179-901c-f37f636d3569 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb8d51663a8a41fe9b58e17f94f52d60', 'request_id': 'req-57801729-10f1-4179-901c-f37f636d3569'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:49.707408 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b6fba-479a-40da-9758-58d6327f8439 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:49.771245 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785f1ec-4270-4e34-bfb6-b06f5bf7d5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:49.771858 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785f1ec-4270-4e34-bfb6-b06f5bf7d5da admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:49.772141 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785f1ec-4270-4e34-bfb6-b06f5bf7d5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:49.772415 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785f1ec-4270-4e34-bfb6-b06f5bf7d5da admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:49.773124 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b785f1ec-4270-4e34-bfb6-b06f5bf7d5da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:49.775058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b785f1ec-4270-4e34-bfb6-b06f5bf7d5da admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:49.787272 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-57801729-10f1-4179-901c-f37f636d3569 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] The token cache is being invalidate because project cb8d51663a8a41fe9b58e17f94f52d60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:49.787642 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-57801729-10f1-4179-901c-f37f636d3569 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:49.788400 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-57801729-10f1-4179-901c-f37f636d3569 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:49.797043 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 999/2004] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:49 2022] DELETE /identity/v3/projects/cb8d51663a8a41fe9b58e17f94f52d60 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:49.907829 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1006/2005] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:47:49 2022] POST /identity/v3/users => generated 301 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:47:49.912132 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57801729-10f1-4179-901c-f37f636d3569 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:49.913800 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-57801729-10f1-4179-901c-f37f636d3569 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:49.927868 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57801729-10f1-4179-901c-f37f636d3569 tempest-VolumeQuotaClassesTest-243787800 tempest-VolumeQuotaClassesTest-243787800-project-admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:49.932857 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e2fee-44bb-45fc-9c8e-2cf0ebf7b480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:49.934071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e2fee-44bb-45fc-9c8e-2cf0ebf7b480 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:49.934071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e2fee-44bb-45fc-9c8e-2cf0ebf7b480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:49.934071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e2fee-44bb-45fc-9c8e-2cf0ebf7b480 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:49.934810 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e2fee-44bb-45fc-9c8e-2cf0ebf7b480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:49.936704 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e2fee-44bb-45fc-9c8e-2cf0ebf7b480 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:49.947482 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1000/2006] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:49.952341 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785f1ec-4270-4e34-bfb6-b06f5bf7d5da admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:49.963015 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785f1ec-4270-4e34-bfb6-b06f5bf7d5da admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:49.969181 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6a637-b8a2-4c13-96db-4ebcddbb1d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:49.969884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6a637-b8a2-4c13-96db-4ebcddbb1d83 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:49.970167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6a637-b8a2-4c13-96db-4ebcddbb1d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:49.970447 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6a637-b8a2-4c13-96db-4ebcddbb1d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:49.971186 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6a637-b8a2-4c13-96db-4ebcddbb1d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:49.972871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6a637-b8a2-4c13-96db-4ebcddbb1d83 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:49.984176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1007/2007] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:49.988635 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e2fee-44bb-45fc-9c8e-2cf0ebf7b480 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:50.000512 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e2fee-44bb-45fc-9c8e-2cf0ebf7b480 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:50.005544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f0bcf8-1855-4161-b30e-80f9da7276cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:50.006447 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0bcf8-1855-4161-b30e-80f9da7276cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:50.006737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0bcf8-1855-4161-b30e-80f9da7276cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:50.007038 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0bcf8-1855-4161-b30e-80f9da7276cf admin admin] PATH_INFO: `/v3/projects/77750133d6024906b47e2d3d8c854dc2/users/98dd498f93394a8388c95b3a1fbccf10/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:50.026867 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f0bcf8-1855-4161-b30e-80f9da7276cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=77750133d6024906b47e2d3d8c854dc2, user_id=98dd498f93394a8388c95b3a1fbccf10, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:50.032513 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f0bcf8-1855-4161-b30e-80f9da7276cf admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:50.057059 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6a637-b8a2-4c13-96db-4ebcddbb1d83 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:50.058833 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b7b6a637-b8a2-4c13-96db-4ebcddbb1d83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:50.072271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6a637-b8a2-4c13-96db-4ebcddbb1d83 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:50.078859 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d2ca9-d73f-437e-b4b1-4c631540513d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:50.079903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d2ca9-d73f-437e-b4b1-4c631540513d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:50.080285 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d2ca9-d73f-437e-b4b1-4c631540513d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:50.080603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d2ca9-d73f-437e-b4b1-4c631540513d admin admin] PATH_INFO: `/v3/users/77a3bb1f73db42759a61495000300612` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:50.082070 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d2ca9-d73f-437e-b4b1-4c631540513d admin admin] RBAC: Authorizing `identity:delete_user(user_id=77a3bb1f73db42759a61495000300612)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:50.084177 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d2ca9-d73f-437e-b4b1-4c631540513d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:50.152761 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:50.167338 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1001/2008] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:49 2022] PUT /identity/v3/projects/77750133d6024906b47e2d3d8c854dc2/users/98dd498f93394a8388c95b3a1fbccf10/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:50.179800 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49dd7d-bcc2-47c1-ad6d-8cffc91eacc1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:50.180095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49dd7d-bcc2-47c1-ad6d-8cffc91eacc1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:50.180378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49dd7d-bcc2-47c1-ad6d-8cffc91eacc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:50.213115 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1d49dd7d-bcc2-47c1-ad6d-8cffc91eacc1 None None] MFA Rules not processed for user `98dd498f93394a8388c95b3a1fbccf10`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:50.214323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1d49dd7d-bcc2-47c1-ad6d-8cffc91eacc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:50.275658 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:50.302549 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1002/2009] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:47:50.307057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49dd7d-bcc2-47c1-ad6d-8cffc91eacc1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:50.357885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e83d2ca9-d73f-437e-b4b1-4c631540513d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77a3bb1f73db42759a61495000300612'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:50.358714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e83d2ca9-d73f-437e-b4b1-4c631540513d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77a3bb1f73db42759a61495000300612'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:50.359854 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e83d2ca9-d73f-437e-b4b1-4c631540513d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77a3bb1f73db42759a61495000300612'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:50.380730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e83d2ca9-d73f-437e-b4b1-4c631540513d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77a3bb1f73db42759a61495000300612'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:50.381508 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49dd7d-bcc2-47c1-ad6d-8cffc91eacc1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:50.387950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75574862-73a9-439f-a33a-01d55525ec12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:50.388543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75574862-73a9-439f-a33a-01d55525ec12 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:50.388837 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75574862-73a9-439f-a33a-01d55525ec12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:50.389208 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75574862-73a9-439f-a33a-01d55525ec12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:50.389897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75574862-73a9-439f-a33a-01d55525ec12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:50.392124 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75574862-73a9-439f-a33a-01d55525ec12 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:50.395777 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e83d2ca9-d73f-437e-b4b1-4c631540513d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77a3bb1f73db42759a61495000300612'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:50.468162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1008/2010] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:50 2022] DELETE /identity/v3/users/77a3bb1f73db42759a61495000300612 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:50.473181 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77616a-ead9-47b9-af28-584cb3a03f90 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:50.476056 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77616a-ead9-47b9-af28-584cb3a03f90 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:50.476739 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77616a-ead9-47b9-af28-584cb3a03f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:50.502416 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-fe77616a-ead9-47b9-af28-584cb3a03f90 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:50.504027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-fe77616a-ead9-47b9-af28-584cb3a03f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:50.507809 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1003/2011] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:47:50 2022] POST /identity/v3/projects => generated 401 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:47:50.513064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75574862-73a9-439f-a33a-01d55525ec12 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:50.515300 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-75574862-73a9-439f-a33a-01d55525ec12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:50.599975 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1009/2012] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:50.600998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77616a-ead9-47b9-af28-584cb3a03f90 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:50.602461 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-fe77616a-ead9-47b9-af28-584cb3a03f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:50.621436 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75574862-73a9-439f-a33a-01d55525ec12 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:50.636432 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f9d22-6873-48f9-8368-3865b0eeadc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:50.638553 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f9d22-6873-48f9-8368-3865b0eeadc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:50.638757 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f9d22-6873-48f9-8368-3865b0eeadc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:50.639042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f9d22-6873-48f9-8368-3865b0eeadc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:50.639944 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343f9d22-6873-48f9-8368-3865b0eeadc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:50.642702 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343f9d22-6873-48f9-8368-3865b0eeadc2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:50.654641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77616a-ead9-47b9-af28-584cb3a03f90 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:50.662342 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f51fcc-f15c-4999-a964-96bcd60026c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:50.663249 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f51fcc-f15c-4999-a964-96bcd60026c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:50.663535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f51fcc-f15c-4999-a964-96bcd60026c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:50.663879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f51fcc-f15c-4999-a964-96bcd60026c8 admin admin] PATH_INFO: `/v3/users/c52e77d8ab0f47e9a60113193a8872e8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:50.666130 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f51fcc-f15c-4999-a964-96bcd60026c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c52e77d8ab0f47e9a60113193a8872e8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:50.669234 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f51fcc-f15c-4999-a964-96bcd60026c8 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:50.704254 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:50.742580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1004/2013] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:47:50 2022] POST /identity/v3/users => generated 307 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:50.744015 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f9d22-6873-48f9-8368-3865b0eeadc2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:50.745074 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-343f9d22-6873-48f9-8368-3865b0eeadc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:50.812927 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f9d22-6873-48f9-8368-3865b0eeadc2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:50.817419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b07d7d-4354-472a-8a28-dc0781feacc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:50.818091 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b07d7d-4354-472a-8a28-dc0781feacc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:50.818374 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b07d7d-4354-472a-8a28-dc0781feacc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:50.818658 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b07d7d-4354-472a-8a28-dc0781feacc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:50.830815 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b07d7d-4354-472a-8a28-dc0781feacc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:50.832711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b07d7d-4354-472a-8a28-dc0781feacc9 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:50.844564 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1005/2014] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:47:50.845601 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b07d7d-4354-472a-8a28-dc0781feacc9 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:50.856069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b07d7d-4354-472a-8a28-dc0781feacc9 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:50.861687 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55d91e-44f6-45db-9ed4-84d013991a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:50.862124 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55d91e-44f6-45db-9ed4-84d013991a8c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:50.862414 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55d91e-44f6-45db-9ed4-84d013991a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:50.862698 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55d91e-44f6-45db-9ed4-84d013991a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:50.863442 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b55d91e-44f6-45db-9ed4-84d013991a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:50.865144 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b55d91e-44f6-45db-9ed4-84d013991a8c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:50.876832 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1006/2015] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:50.881455 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55d91e-44f6-45db-9ed4-84d013991a8c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:50.899969 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55d91e-44f6-45db-9ed4-84d013991a8c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:50.906037 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4093b6b-165d-4158-9c0c-7462f2df3a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:50.906977 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4093b6b-165d-4158-9c0c-7462f2df3a9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:50.907244 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4093b6b-165d-4158-9c0c-7462f2df3a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:50.907970 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4093b6b-165d-4158-9c0c-7462f2df3a9e admin admin] PATH_INFO: `/v3/projects/6bdffeef468f49d4bf618d12d3c68389/users/9ab5dfdb90794e86ab07feb630343bb1/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:50.926175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c7f51fcc-f15c-4999-a964-96bcd60026c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c52e77d8ab0f47e9a60113193a8872e8'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:50.928004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c7f51fcc-f15c-4999-a964-96bcd60026c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c52e77d8ab0f47e9a60113193a8872e8'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:50.929190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c7f51fcc-f15c-4999-a964-96bcd60026c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c52e77d8ab0f47e9a60113193a8872e8'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:50.933102 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4093b6b-165d-4158-9c0c-7462f2df3a9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bdffeef468f49d4bf618d12d3c68389, user_id=9ab5dfdb90794e86ab07feb630343bb1, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:50.940219 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4093b6b-165d-4158-9c0c-7462f2df3a9e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:50.956752 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c7f51fcc-f15c-4999-a964-96bcd60026c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c52e77d8ab0f47e9a60113193a8872e8'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:50.972121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c7f51fcc-f15c-4999-a964-96bcd60026c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c52e77d8ab0f47e9a60113193a8872e8'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:50.998625 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:51.027097 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1007/2016] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:50 2022] PUT /identity/v3/projects/6bdffeef468f49d4bf618d12d3c68389/users/9ab5dfdb90794e86ab07feb630343bb1/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:51.031888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4093b6b-165d-4158-9c0c-7462f2df3a9e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:51.033674 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f4093b6b-165d-4158-9c0c-7462f2df3a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:51.107659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1010/2017] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:50 2022] DELETE /identity/v3/users/c52e77d8ab0f47e9a60113193a8872e8 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:51.112713 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee8408-e92b-4923-b5ff-7c5647392606 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:51.113081 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee8408-e92b-4923-b5ff-7c5647392606 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:51.113365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4093b6b-165d-4158-9c0c-7462f2df3a9e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:51.113479 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee8408-e92b-4923-b5ff-7c5647392606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:51.138931 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-fdee8408-e92b-4923-b5ff-7c5647392606 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:51.144909 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-fdee8408-e92b-4923-b5ff-7c5647392606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:51.196205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eccc5c-7f85-4934-a748-4b9a12337e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:51.197104 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eccc5c-7f85-4934-a748-4b9a12337e0e admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:51.197398 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eccc5c-7f85-4934-a748-4b9a12337e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:51.197686 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eccc5c-7f85-4934-a748-4b9a12337e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:51.198451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eccc5c-7f85-4934-a748-4b9a12337e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:51.200222 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eccc5c-7f85-4934-a748-4b9a12337e0e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:51.214064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1008/2018] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:47:51.220225 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eccc5c-7f85-4934-a748-4b9a12337e0e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:51.242283 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eccc5c-7f85-4934-a748-4b9a12337e0e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:51.248163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f19ef-a95d-469b-9d6f-7d92ec927a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:51.249311 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f19ef-a95d-469b-9d6f-7d92ec927a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:51.249668 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f19ef-a95d-469b-9d6f-7d92ec927a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:51.250022 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f19ef-a95d-469b-9d6f-7d92ec927a27 admin admin] PATH_INFO: `/v3/projects/6bdffeef468f49d4bf618d12d3c68389/users/9ab5dfdb90794e86ab07feb630343bb1/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:51.252549 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f19ef-a95d-469b-9d6f-7d92ec927a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bdffeef468f49d4bf618d12d3c68389, user_id=9ab5dfdb90794e86ab07feb630343bb1, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:51.255706 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f19ef-a95d-469b-9d6f-7d92ec927a27 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:51.272016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1009/2019] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:51 2022] PUT /identity/v3/projects/6bdffeef468f49d4bf618d12d3c68389/users/9ab5dfdb90794e86ab07feb630343bb1/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:51.281028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34899edb-b862-4f71-a1f0-f09a307d0068 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:51.281823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34899edb-b862-4f71-a1f0-f09a307d0068 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:51.282117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34899edb-b862-4f71-a1f0-f09a307d0068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:51.283194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1011/2020] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:47:51.299589 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee8408-e92b-4923-b5ff-7c5647392606 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:51.301543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-fdee8408-e92b-4923-b5ff-7c5647392606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:51.316774 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-34899edb-b862-4f71-a1f0-f09a307d0068 None None] MFA Rules not processed for user `9ab5dfdb90794e86ab07feb630343bb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:51.319036 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-34899edb-b862-4f71-a1f0-f09a307d0068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:51.343326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:51.378950 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee8408-e92b-4923-b5ff-7c5647392606 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:51.384440 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:51.438220 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3560238-61ef-4829-8c9f-ea8f7923e4a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:51.438765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3560238-61ef-4829-8c9f-ea8f7923e4a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:51.439132 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3560238-61ef-4829-8c9f-ea8f7923e4a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:51.439407 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3560238-61ef-4829-8c9f-ea8f7923e4a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:51.458343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1010/2021] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:51 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:47:51.472330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34899edb-b862-4f71-a1f0-f09a307d0068 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:51.474123 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-34899edb-b862-4f71-a1f0-f09a307d0068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:51.529917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3560238-61ef-4829-8c9f-ea8f7923e4a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:51.531641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3560238-61ef-4829-8c9f-ea8f7923e4a1 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:51.545281 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1012/2022] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:47:51.573025 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34899edb-b862-4f71-a1f0-f09a307d0068 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:51.576652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af1b90-15bd-4aa2-bad7-ab8abebee04c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:51.578113 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af1b90-15bd-4aa2-bad7-ab8abebee04c service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:51.578405 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af1b90-15bd-4aa2-bad7-ab8abebee04c service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:51.578925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af1b90-15bd-4aa2-bad7-ab8abebee04c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:51.593144 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af1b90-15bd-4aa2-bad7-ab8abebee04c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:51.594696 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af1b90-15bd-4aa2-bad7-ab8abebee04c service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:51.606523 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1011/2023] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:51 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:47:51.642247 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3560238-61ef-4829-8c9f-ea8f7923e4a1 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:51.644160 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b3560238-61ef-4829-8c9f-ea8f7923e4a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:51.665868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3560238-61ef-4829-8c9f-ea8f7923e4a1 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:51.677867 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca8fb7-7862-416b-9af7-0f8e65853a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:51.680057 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca8fb7-7862-416b-9af7-0f8e65853a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:51.680875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca8fb7-7862-416b-9af7-0f8e65853a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:51.684269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca8fb7-7862-416b-9af7-0f8e65853a3f admin admin] PATH_INFO: `/v3/projects/79d2e618d9cd4648bc71619b974a0fa3` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:51.688453 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca8fb7-7862-416b-9af7-0f8e65853a3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=79d2e618d9cd4648bc71619b974a0fa3)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:51.693792 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca8fb7-7862-416b-9af7-0f8e65853a3f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:51.911138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af1b90-15bd-4aa2-bad7-ab8abebee04c service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:51.920710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-abca8fb7-7862-416b-9af7-0f8e65853a3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79d2e618d9cd4648bc71619b974a0fa3', 'request_id': 'req-abca8fb7-7862-416b-9af7-0f8e65853a3f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:51.928769 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af1b90-15bd-4aa2-bad7-ab8abebee04c service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:51.943829 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-abca8fb7-7862-416b-9af7-0f8e65853a3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79d2e618d9cd4648bc71619b974a0fa3', 'request_id': 'req-abca8fb7-7862-416b-9af7-0f8e65853a3f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:51.953238 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-abca8fb7-7862-416b-9af7-0f8e65853a3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79d2e618d9cd4648bc71619b974a0fa3', 'request_id': 'req-abca8fb7-7862-416b-9af7-0f8e65853a3f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:52.013345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4397a-90f6-49fa-ac55-9ec42126d55b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:52.014162 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4397a-90f6-49fa-ac55-9ec42126d55b service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:52.014367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4397a-90f6-49fa-ac55-9ec42126d55b service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:52.014653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4397a-90f6-49fa-ac55-9ec42126d55b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:52.016897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c9f4397a-90f6-49fa-ac55-9ec42126d55b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:52.040037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-abca8fb7-7862-416b-9af7-0f8e65853a3f admin admin] The token cache is being invalidate because project 79d2e618d9cd4648bc71619b974a0fa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:52.041136 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-abca8fb7-7862-416b-9af7-0f8e65853a3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:52.042125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-abca8fb7-7862-416b-9af7-0f8e65853a3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:52.047598 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1013/2024] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:51 2022] DELETE /identity/v3/projects/79d2e618d9cd4648bc71619b974a0fa3 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:52.052333 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d84ef-1e9d-4229-aa04-7103f0f8ce7a None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:52.053230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d84ef-1e9d-4229-aa04-7103f0f8ce7a None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:52.053604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d84ef-1e9d-4229-aa04-7103f0f8ce7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:52.077390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-159d84ef-1e9d-4229-aa04-7103f0f8ce7a None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:52.079000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-159d84ef-1e9d-4229-aa04-7103f0f8ce7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:52.088655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:52.109699 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f4397a-90f6-49fa-ac55-9ec42126d55b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:52.112238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f4397a-90f6-49fa-ac55-9ec42126d55b service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:52.129891 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1012/2025] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:51 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:47:52.148295 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:52.188141 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1014/2026] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:52.189943 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91175d-760e-4dca-a82f-e7ab99a3f1d1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:52.190323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91175d-760e-4dca-a82f-e7ab99a3f1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:52.190588 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91175d-760e-4dca-a82f-e7ab99a3f1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:52.211651 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8f91175d-760e-4dca-a82f-e7ab99a3f1d1 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:52.212965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8f91175d-760e-4dca-a82f-e7ab99a3f1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:52.224285 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1013/2027] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:52.234923 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d84ef-1e9d-4229-aa04-7103f0f8ce7a None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:52.246137 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d84ef-1e9d-4229-aa04-7103f0f8ce7a None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:52.253064 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a57f5-2a29-4183-8939-3b02853d5eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:52.253749 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a57f5-2a29-4183-8939-3b02853d5eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:52.254023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a57f5-2a29-4183-8939-3b02853d5eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:52.254297 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a57f5-2a29-4183-8939-3b02853d5eb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:52.255040 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a57f5-2a29-4183-8939-3b02853d5eb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:47:52.255334 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a57f5-2a29-4183-8939-3b02853d5eb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:52.256992 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a57f5-2a29-4183-8939-3b02853d5eb2 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:52.270606 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1015/2028] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:47:52.275538 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91175d-760e-4dca-a82f-e7ab99a3f1d1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:52.288473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91175d-760e-4dca-a82f-e7ab99a3f1d1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:52.295276 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a57f5-2a29-4183-8939-3b02853d5eb2 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:52.297826 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-810a57f5-2a29-4183-8939-3b02853d5eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:52.307579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6de85-5289-4302-b1a1-765787387fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:52.308499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6de85-5289-4302-b1a1-765787387fcf admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:52.308583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6de85-5289-4302-b1a1-765787387fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:52.308953 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6de85-5289-4302-b1a1-765787387fcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:52.310704 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6de85-5289-4302-b1a1-765787387fcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:52.312814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6de85-5289-4302-b1a1-765787387fcf admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:52.318245 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a57f5-2a29-4183-8939-3b02853d5eb2 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:52.323994 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecd7db-e98b-4813-b83d-831d0e706565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:52.325105 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecd7db-e98b-4813-b83d-831d0e706565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:52.325380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecd7db-e98b-4813-b83d-831d0e706565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:52.325657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ecd7db-e98b-4813-b83d-831d0e706565 admin admin] PATH_INFO: `/v3/users/023824b9e78242bd9d10aeb3a52c6d14` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:52.327051 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ecd7db-e98b-4813-b83d-831d0e706565 admin admin] RBAC: Authorizing `identity:delete_user(user_id=023824b9e78242bd9d10aeb3a52c6d14)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:52.329343 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ecd7db-e98b-4813-b83d-831d0e706565 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:52.436814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1014/2029] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:47:52 2022] POST /identity/v3/projects => generated 371 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:52.436814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6de85-5289-4302-b1a1-765787387fcf admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:52.438537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b5f6de85-5289-4302-b1a1-765787387fcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:52.516180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6de85-5289-4302-b1a1-765787387fcf admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:52.521250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ee729-30b5-4204-9279-5400376a1d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:52.522520 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ee729-30b5-4204-9279-5400376a1d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:52.522820 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ee729-30b5-4204-9279-5400376a1d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:52.523104 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ee729-30b5-4204-9279-5400376a1d48 admin admin] PATH_INFO: `/v3/projects/d3ce0c562bf145939da6703b00c2b99b` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:52.524398 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ee729-30b5-4204-9279-5400376a1d48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3ce0c562bf145939da6703b00c2b99b)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:52.526519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ee729-30b5-4204-9279-5400376a1d48 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:52.601491 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-03ecd7db-e98b-4813-b83d-831d0e706565 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '023824b9e78242bd9d10aeb3a52c6d14'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:52.602242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-03ecd7db-e98b-4813-b83d-831d0e706565 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '023824b9e78242bd9d10aeb3a52c6d14'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:52.602923 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-03ecd7db-e98b-4813-b83d-831d0e706565 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '023824b9e78242bd9d10aeb3a52c6d14'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:52.618689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-03ecd7db-e98b-4813-b83d-831d0e706565 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '023824b9e78242bd9d10aeb3a52c6d14'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:52.628912 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b67ee729-30b5-4204-9279-5400376a1d48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3ce0c562bf145939da6703b00c2b99b', 'request_id': 'req-b67ee729-30b5-4204-9279-5400376a1d48'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:52.632458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-03ecd7db-e98b-4813-b83d-831d0e706565 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '023824b9e78242bd9d10aeb3a52c6d14'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:52.636605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b67ee729-30b5-4204-9279-5400376a1d48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3ce0c562bf145939da6703b00c2b99b', 'request_id': 'req-b67ee729-30b5-4204-9279-5400376a1d48'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:52.645208 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b67ee729-30b5-4204-9279-5400376a1d48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3ce0c562bf145939da6703b00c2b99b', 'request_id': 'req-b67ee729-30b5-4204-9279-5400376a1d48'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:52.723567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1016/2030] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:52 2022] DELETE /identity/v3/users/023824b9e78242bd9d10aeb3a52c6d14 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:52.724468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecd7db-e98b-4813-b83d-831d0e706565 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:52.725784 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-03ecd7db-e98b-4813-b83d-831d0e706565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:52.801670 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ecd7db-e98b-4813-b83d-831d0e706565 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:52.807131 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2464df-bca1-4357-a345-06d24b385b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:52.813816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2464df-bca1-4357-a345-06d24b385b69 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:52.813816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2464df-bca1-4357-a345-06d24b385b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:52.813816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2464df-bca1-4357-a345-06d24b385b69 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:52.813816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2464df-bca1-4357-a345-06d24b385b69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:52.813816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2464df-bca1-4357-a345-06d24b385b69 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:52.837212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b67ee729-30b5-4204-9279-5400376a1d48 admin admin] The token cache is being invalidate because project d3ce0c562bf145939da6703b00c2b99b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:52.846838 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b67ee729-30b5-4204-9279-5400376a1d48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:52.846838 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b67ee729-30b5-4204-9279-5400376a1d48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:52.846838 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1015/2031] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:52 2022] DELETE /identity/v3/projects/d3ce0c562bf145939da6703b00c2b99b => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:52.846838 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ee729-30b5-4204-9279-5400376a1d48 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:52.846838 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b67ee729-30b5-4204-9279-5400376a1d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:52.914610 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:52.936076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ee729-30b5-4204-9279-5400376a1d48 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:52.944859 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be752775-9c5c-4060-8916-d84d7ad5d25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:52.946474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be752775-9c5c-4060-8916-d84d7ad5d25f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:52.946937 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be752775-9c5c-4060-8916-d84d7ad5d25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:52.947432 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be752775-9c5c-4060-8916-d84d7ad5d25f admin admin] PATH_INFO: `/v3/users/51edb1a774ed45d1ba1422fb456ad927` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:52.950554 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be752775-9c5c-4060-8916-d84d7ad5d25f admin admin] RBAC: Authorizing `identity:delete_user(user_id=51edb1a774ed45d1ba1422fb456ad927)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:52.955741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be752775-9c5c-4060-8916-d84d7ad5d25f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:52.967609 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1017/2032] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:47:52 2022] POST /identity/v3/users => generated 286 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:52.973057 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2464df-bca1-4357-a345-06d24b385b69 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:52.978324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3a2464df-bca1-4357-a345-06d24b385b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:53.000485 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2464df-bca1-4357-a345-06d24b385b69 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:53.008904 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d92dec-d940-4b12-8d7d-53b038d976ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:53.009727 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d92dec-d940-4b12-8d7d-53b038d976ba admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:53.010016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d92dec-d940-4b12-8d7d-53b038d976ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:53.010301 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d92dec-d940-4b12-8d7d-53b038d976ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:53.011084 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d92dec-d940-4b12-8d7d-53b038d976ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:53.014980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d92dec-d940-4b12-8d7d-53b038d976ba admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:53.034510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1018/2033] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:47:53.040975 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d92dec-d940-4b12-8d7d-53b038d976ba admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:53.063448 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d92dec-d940-4b12-8d7d-53b038d976ba admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:53.068877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48c0a9-a3b7-4103-87a4-41b027fad6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:53.069568 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48c0a9-a3b7-4103-87a4-41b027fad6bd admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:53.069853 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48c0a9-a3b7-4103-87a4-41b027fad6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:53.070129 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48c0a9-a3b7-4103-87a4-41b027fad6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:53.070927 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae48c0a9-a3b7-4103-87a4-41b027fad6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:53.072619 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae48c0a9-a3b7-4103-87a4-41b027fad6bd admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:53.093822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1019/2034] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:53.096090 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48c0a9-a3b7-4103-87a4-41b027fad6bd admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:53.098420 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ae48c0a9-a3b7-4103-87a4-41b027fad6bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:53.113979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48c0a9-a3b7-4103-87a4-41b027fad6bd admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:53.127820 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674fa9d-d30b-4cd6-8860-34353a984d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:53.128961 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674fa9d-d30b-4cd6-8860-34353a984d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:53.129350 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674fa9d-d30b-4cd6-8860-34353a984d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:53.129642 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674fa9d-d30b-4cd6-8860-34353a984d86 admin admin] PATH_INFO: `/v3/projects/4f9e24761e274293abd9f9cb362bb4a3` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:53.131028 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674fa9d-d30b-4cd6-8860-34353a984d86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f9e24761e274293abd9f9cb362bb4a3)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:53.133544 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674fa9d-d30b-4cd6-8860-34353a984d86 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:53.186257 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-be752775-9c5c-4060-8916-d84d7ad5d25f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51edb1a774ed45d1ba1422fb456ad927'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:53.186942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-be752775-9c5c-4060-8916-d84d7ad5d25f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51edb1a774ed45d1ba1422fb456ad927'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:53.187557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-be752775-9c5c-4060-8916-d84d7ad5d25f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51edb1a774ed45d1ba1422fb456ad927'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:53.202197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-be752775-9c5c-4060-8916-d84d7ad5d25f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51edb1a774ed45d1ba1422fb456ad927'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:53.210201 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-be752775-9c5c-4060-8916-d84d7ad5d25f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51edb1a774ed45d1ba1422fb456ad927'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:53.261303 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9674fa9d-d30b-4cd6-8860-34353a984d86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f9e24761e274293abd9f9cb362bb4a3', 'request_id': 'req-9674fa9d-d30b-4cd6-8860-34353a984d86'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:53.267063 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9674fa9d-d30b-4cd6-8860-34353a984d86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f9e24761e274293abd9f9cb362bb4a3', 'request_id': 'req-9674fa9d-d30b-4cd6-8860-34353a984d86'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:53.275116 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9674fa9d-d30b-4cd6-8860-34353a984d86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f9e24761e274293abd9f9cb362bb4a3', 'request_id': 'req-9674fa9d-d30b-4cd6-8860-34353a984d86'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:53.319052 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1016/2035] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:52 2022] DELETE /identity/v3/users/51edb1a774ed45d1ba1422fb456ad927 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:53.320137 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be752775-9c5c-4060-8916-d84d7ad5d25f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:53.322195 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-be752775-9c5c-4060-8916-d84d7ad5d25f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:53.388544 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9674fa9d-d30b-4cd6-8860-34353a984d86 admin admin] The token cache is being invalidate because project 4f9e24761e274293abd9f9cb362bb4a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:53.389581 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9674fa9d-d30b-4cd6-8860-34353a984d86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:53.390737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9674fa9d-d30b-4cd6-8860-34353a984d86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:53.393372 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1020/2036] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:53 2022] DELETE /identity/v3/projects/4f9e24761e274293abd9f9cb362bb4a3 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:53.398004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674fa9d-d30b-4cd6-8860-34353a984d86 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:53.400724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9674fa9d-d30b-4cd6-8860-34353a984d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:53.411792 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:53.427558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be752775-9c5c-4060-8916-d84d7ad5d25f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:53.434982 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e8c54-369f-42b4-b94e-d051c1614ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:53.436338 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e8c54-369f-42b4-b94e-d051c1614ff5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:53.437410 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e8c54-369f-42b4-b94e-d051c1614ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:53.437670 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e8c54-369f-42b4-b94e-d051c1614ff5 admin admin] PATH_INFO: `/v3/projects/48ac5e93576143899d5906ac93229e24/users/5401571f04fb464fbfc7201732508be7/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:53.455723 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e8c54-369f-42b4-b94e-d051c1614ff5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48ac5e93576143899d5906ac93229e24, user_id=5401571f04fb464fbfc7201732508be7, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:53.458597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e8c54-369f-42b4-b94e-d051c1614ff5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:53.488315 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674fa9d-d30b-4cd6-8860-34353a984d86 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:53.493357 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a9327-cc43-42c8-a841-0717cf3575b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:53.494251 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a9327-cc43-42c8-a841-0717cf3575b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:53.494532 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a9327-cc43-42c8-a841-0717cf3575b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:53.494807 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a9327-cc43-42c8-a841-0717cf3575b7 admin admin] PATH_INFO: `/v3/users/80f62eadb2404a74bdffb4adba33f1f7` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:53.496129 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a9327-cc43-42c8-a841-0717cf3575b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80f62eadb2404a74bdffb4adba33f1f7)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:53.498586 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a9327-cc43-42c8-a841-0717cf3575b7 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:53.564704 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1017/2037] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:53 2022] PUT /identity/v3/projects/48ac5e93576143899d5906ac93229e24/users/5401571f04fb464fbfc7201732508be7/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:53.588650 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be6c33-33e7-483a-bcb4-99649f7b78c3 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:53.589166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be6c33-33e7-483a-bcb4-99649f7b78c3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:53.589653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be6c33-33e7-483a-bcb4-99649f7b78c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:53.645053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f9be6c33-33e7-483a-bcb4-99649f7b78c3 None None] MFA Rules not processed for user `5401571f04fb464fbfc7201732508be7`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:53.647332 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f9be6c33-33e7-483a-bcb4-99649f7b78c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:53.703252 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f60a9327-cc43-42c8-a841-0717cf3575b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80f62eadb2404a74bdffb4adba33f1f7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:53.704128 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f60a9327-cc43-42c8-a841-0717cf3575b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80f62eadb2404a74bdffb4adba33f1f7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:53.704810 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f60a9327-cc43-42c8-a841-0717cf3575b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80f62eadb2404a74bdffb4adba33f1f7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:53.728424 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:53.728741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f60a9327-cc43-42c8-a841-0717cf3575b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80f62eadb2404a74bdffb4adba33f1f7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:53.747971 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f60a9327-cc43-42c8-a841-0717cf3575b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80f62eadb2404a74bdffb4adba33f1f7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:53.769440 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1018/2038] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:53.784561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be6c33-33e7-483a-bcb4-99649f7b78c3 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:53.786161 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f9be6c33-33e7-483a-bcb4-99649f7b78c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:53.865955 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be6c33-33e7-483a-bcb4-99649f7b78c3 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:53.870180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce304e4f-498b-4553-b99a-f5e82f6d6f08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:53.870861 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce304e4f-498b-4553-b99a-f5e82f6d6f08 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:53.871148 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce304e4f-498b-4553-b99a-f5e82f6d6f08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:53.871433 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce304e4f-498b-4553-b99a-f5e82f6d6f08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:53.880146 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1021/2039] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:53 2022] DELETE /identity/v3/users/80f62eadb2404a74bdffb4adba33f1f7 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:53.881614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce304e4f-498b-4553-b99a-f5e82f6d6f08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:53.883095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce304e4f-498b-4553-b99a-f5e82f6d6f08 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:53.896824 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1019/2040] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:47:53.926223 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a9327-cc43-42c8-a841-0717cf3575b7 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:53.927913 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f60a9327-cc43-42c8-a841-0717cf3575b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:53.935896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce304e4f-498b-4553-b99a-f5e82f6d6f08 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:53.938026 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ce304e4f-498b-4553-b99a-f5e82f6d6f08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:54.017998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a9327-cc43-42c8-a841-0717cf3575b7 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:54.024645 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2710e-29bd-4e02-a3e2-004c3438dcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:54.025216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2710e-29bd-4e02-a3e2-004c3438dcd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:54.025502 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2710e-29bd-4e02-a3e2-004c3438dcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:54.025780 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2710e-29bd-4e02-a3e2-004c3438dcd9 admin admin] PATH_INFO: `/v3/projects/e66ea819617644b18b43ee4565c28b4e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:54.030522 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2710e-29bd-4e02-a3e2-004c3438dcd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e66ea819617644b18b43ee4565c28b4e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:54.033103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2710e-29bd-4e02-a3e2-004c3438dcd9 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:54.051390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:54.093308 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce304e4f-498b-4553-b99a-f5e82f6d6f08 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:54.098729 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4ee30-b28a-4fce-9e39-1162f7de2b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:54.099712 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4ee30-b28a-4fce-9e39-1162f7de2b77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:54.099999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4ee30-b28a-4fce-9e39-1162f7de2b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:54.100280 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4ee30-b28a-4fce-9e39-1162f7de2b77 admin admin] PATH_INFO: `/v3/users/5401571f04fb464fbfc7201732508be7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:54.101641 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4ee30-b28a-4fce-9e39-1162f7de2b77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5401571f04fb464fbfc7201732508be7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:54.103783 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4ee30-b28a-4fce-9e39-1162f7de2b77 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:54.281180 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e2a2710e-29bd-4e02-a3e2-004c3438dcd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e66ea819617644b18b43ee4565c28b4e', 'request_id': 'req-e2a2710e-29bd-4e02-a3e2-004c3438dcd9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:54.290167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e2a2710e-29bd-4e02-a3e2-004c3438dcd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e66ea819617644b18b43ee4565c28b4e', 'request_id': 'req-e2a2710e-29bd-4e02-a3e2-004c3438dcd9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:54.300169 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e2a2710e-29bd-4e02-a3e2-004c3438dcd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e66ea819617644b18b43ee4565c28b4e', 'request_id': 'req-e2a2710e-29bd-4e02-a3e2-004c3438dcd9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:54.398946 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e2a2710e-29bd-4e02-a3e2-004c3438dcd9 admin admin] The token cache is being invalidate because project e66ea819617644b18b43ee4565c28b4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:54.399606 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e2a2710e-29bd-4e02-a3e2-004c3438dcd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:54.400049 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e2a2710e-29bd-4e02-a3e2-004c3438dcd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:54.402000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1022/2041] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:53 2022] DELETE /identity/v3/projects/e66ea819617644b18b43ee4565c28b4e => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:54.403606 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2710e-29bd-4e02-a3e2-004c3438dcd9 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:54.405243 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e2a2710e-29bd-4e02-a3e2-004c3438dcd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:54.416049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a9d4ee30-b28a-4fce-9e39-1162f7de2b77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5401571f04fb464fbfc7201732508be7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:54.417027 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a9d4ee30-b28a-4fce-9e39-1162f7de2b77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5401571f04fb464fbfc7201732508be7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:54.417711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a9d4ee30-b28a-4fce-9e39-1162f7de2b77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5401571f04fb464fbfc7201732508be7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:54.437179 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a9d4ee30-b28a-4fce-9e39-1162f7de2b77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5401571f04fb464fbfc7201732508be7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:54.441101 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a9d4ee30-b28a-4fce-9e39-1162f7de2b77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5401571f04fb464fbfc7201732508be7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:54.492181 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2710e-29bd-4e02-a3e2-004c3438dcd9 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:54.499429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a33915-a855-4c3c-9a7c-e8c9ef9ae1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:54.500984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a33915-a855-4c3c-9a7c-e8c9ef9ae1bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:54.501314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a33915-a855-4c3c-9a7c-e8c9ef9ae1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:54.501668 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a33915-a855-4c3c-9a7c-e8c9ef9ae1bf admin admin] PATH_INFO: `/v3/projects/ab6ce9a4b32f4edfbd9c40da74938006` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:54.503067 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a33915-a855-4c3c-9a7c-e8c9ef9ae1bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab6ce9a4b32f4edfbd9c40da74938006)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:54.505079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a33915-a855-4c3c-9a7c-e8c9ef9ae1bf admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:54.576527 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1020/2042] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:53 2022] DELETE /identity/v3/users/5401571f04fb464fbfc7201732508be7 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:54.578194 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19db79a-f6ee-40fc-a9e1-2a8ea9696a9e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:54.578537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19db79a-f6ee-40fc-a9e1-2a8ea9696a9e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:54.578817 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19db79a-f6ee-40fc-a9e1-2a8ea9696a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:54.586052 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:54.600774 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f19db79a-f6ee-40fc-a9e1-2a8ea9696a9e None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:54.602017 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f19db79a-f6ee-40fc-a9e1-2a8ea9696a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:54.670325 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f8a33915-a855-4c3c-9a7c-e8c9ef9ae1bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab6ce9a4b32f4edfbd9c40da74938006', 'request_id': 'req-f8a33915-a855-4c3c-9a7c-e8c9ef9ae1bf'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:54.677711 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f8a33915-a855-4c3c-9a7c-e8c9ef9ae1bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab6ce9a4b32f4edfbd9c40da74938006', 'request_id': 'req-f8a33915-a855-4c3c-9a7c-e8c9ef9ae1bf'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:54.685414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f8a33915-a855-4c3c-9a7c-e8c9ef9ae1bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab6ce9a4b32f4edfbd9c40da74938006', 'request_id': 'req-f8a33915-a855-4c3c-9a7c-e8c9ef9ae1bf'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:54.700782 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1021/2043] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:54.700782 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0f054-9362-4c98-b713-e1a0d47c7f66 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:54.700782 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0f054-9362-4c98-b713-e1a0d47c7f66 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:54.700782 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0f054-9362-4c98-b713-e1a0d47c7f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:54.722812 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b2e0f054-9362-4c98-b713-e1a0d47c7f66 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:54.723998 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b2e0f054-9362-4c98-b713-e1a0d47c7f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:54.796070 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1022/2044] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:54.798295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428cb85-9935-4054-920e-434dbf4d4919 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:54.798787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428cb85-9935-4054-920e-434dbf4d4919 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:54.799392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428cb85-9935-4054-920e-434dbf4d4919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:54.821666 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f8a33915-a855-4c3c-9a7c-e8c9ef9ae1bf admin admin] The token cache is being invalidate because project ab6ce9a4b32f4edfbd9c40da74938006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:54.822287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f8a33915-a855-4c3c-9a7c-e8c9ef9ae1bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:54.823090 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f8a33915-a855-4c3c-9a7c-e8c9ef9ae1bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:54.829886 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1023/2045] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:54 2022] DELETE /identity/v3/projects/ab6ce9a4b32f4edfbd9c40da74938006 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:54.829886 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a33915-a855-4c3c-9a7c-e8c9ef9ae1bf admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:54.829886 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f8a33915-a855-4c3c-9a7c-e8c9ef9ae1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:54.835600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0428cb85-9935-4054-920e-434dbf4d4919 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:54.837363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0428cb85-9935-4054-920e-434dbf4d4919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:54.852586 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1023/2046] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:54.854494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95340a94-1207-4c5b-8cfe-00172bceced0 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:54.855157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95340a94-1207-4c5b-8cfe-00172bceced0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:54.856250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95340a94-1207-4c5b-8cfe-00172bceced0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:54.890386 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-95340a94-1207-4c5b-8cfe-00172bceced0 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:54.892741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-95340a94-1207-4c5b-8cfe-00172bceced0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:54.904189 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a33915-a855-4c3c-9a7c-e8c9ef9ae1bf admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:54.904790 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1024/2047] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:54.905788 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95340a94-1207-4c5b-8cfe-00172bceced0 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:54.914081 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379634f-6a30-4cc5-b934-7b8a77ab3043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:54.914902 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379634f-6a30-4cc5-b934-7b8a77ab3043 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:54.915348 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379634f-6a30-4cc5-b934-7b8a77ab3043 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:54.915624 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379634f-6a30-4cc5-b934-7b8a77ab3043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:54.917756 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6379634f-6a30-4cc5-b934-7b8a77ab3043 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:54.917895 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95340a94-1207-4c5b-8cfe-00172bceced0 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:54.922911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ec9cb-bf89-49e8-8690-8f146a5f8c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:54.923688 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ec9cb-bf89-49e8-8690-8f146a5f8c7d admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:54.924235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ec9cb-bf89-49e8-8690-8f146a5f8c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:54.924509 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ec9cb-bf89-49e8-8690-8f146a5f8c7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:54.925255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ec9cb-bf89-49e8-8690-8f146a5f8c7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:47:54.925567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ec9cb-bf89-49e8-8690-8f146a5f8c7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:54.927288 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ec9cb-bf89-49e8-8690-8f146a5f8c7d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:54.930987 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379634f-6a30-4cc5-b934-7b8a77ab3043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:54.935868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379634f-6a30-4cc5-b934-7b8a77ab3043 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:54.946014 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1025/2048] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:47:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:47:54.946014 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7931876-beb3-4f8f-86f6-e2c7374fad68 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:54.946014 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7931876-beb3-4f8f-86f6-e2c7374fad68 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:54.946014 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7931876-beb3-4f8f-86f6-e2c7374fad68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:54.947619 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1024/2049] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:47:54.949158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379634f-6a30-4cc5-b934-7b8a77ab3043 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:54.967713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f7931876-beb3-4f8f-86f6-e2c7374fad68 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:54.969053 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379634f-6a30-4cc5-b934-7b8a77ab3043 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:54.969223 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f7931876-beb3-4f8f-86f6-e2c7374fad68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:54.977285 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634e191-cfbb-4d3a-b436-1dc1a4599f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:54.978459 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634e191-cfbb-4d3a-b436-1dc1a4599f86 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:54.984049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1026/2050] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:54.984250 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634e191-cfbb-4d3a-b436-1dc1a4599f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:54.984250 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634e191-cfbb-4d3a-b436-1dc1a4599f86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:54.984250 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634e191-cfbb-4d3a-b436-1dc1a4599f86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:54.987797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7931876-beb3-4f8f-86f6-e2c7374fad68 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:54.988049 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634e191-cfbb-4d3a-b436-1dc1a4599f86 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:54.995718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7931876-beb3-4f8f-86f6-e2c7374fad68 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:55.004116 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c860ad1-d3cf-4230-9fc9-4fee0d21bbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:55.005262 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c860ad1-d3cf-4230-9fc9-4fee0d21bbd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:55.005804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c860ad1-d3cf-4230-9fc9-4fee0d21bbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:55.006304 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c860ad1-d3cf-4230-9fc9-4fee0d21bbd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:55.008498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c860ad1-d3cf-4230-9fc9-4fee0d21bbd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:47:55.009070 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c860ad1-d3cf-4230-9fc9-4fee0d21bbd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:55.012358 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c860ad1-d3cf-4230-9fc9-4fee0d21bbd3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:55.035716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1027/2051] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:47:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:47:55.041136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c860ad1-d3cf-4230-9fc9-4fee0d21bbd3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:55.052909 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c860ad1-d3cf-4230-9fc9-4fee0d21bbd3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:55.058159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7a044-2ea0-4062-8774-065939f12246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:55.058971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7a044-2ea0-4062-8774-065939f12246 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:55.059258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7a044-2ea0-4062-8774-065939f12246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:55.059537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7a044-2ea0-4062-8774-065939f12246 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:55.060443 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7a044-2ea0-4062-8774-065939f12246 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:55.062387 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7a044-2ea0-4062-8774-065939f12246 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:55.098441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1025/2052] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:47:54 2022] POST /identity/v3/projects => generated 367 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:55.102861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634e191-cfbb-4d3a-b436-1dc1a4599f86 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:55.150970 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:55.194535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634e191-cfbb-4d3a-b436-1dc1a4599f86 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:55.195006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1028/2053] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:47:55 2022] POST /identity/v3/projects => generated 375 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:55.196692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7a044-2ea0-4062-8774-065939f12246 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:55.306426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f296597-c3c5-411d-93de-24182e8fb918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:55.308573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f296597-c3c5-411d-93de-24182e8fb918 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:55.310050 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f296597-c3c5-411d-93de-24182e8fb918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:55.315354 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f296597-c3c5-411d-93de-24182e8fb918 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:55.315354 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f296597-c3c5-411d-93de-24182e8fb918 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:55.320655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f296597-c3c5-411d-93de-24182e8fb918 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:55.337697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7a044-2ea0-4062-8774-065939f12246 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:55.343633 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:55.352218 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2adfa-393a-4f7d-afec-55f6674aa074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:55.354344 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2adfa-393a-4f7d-afec-55f6674aa074 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:55.354792 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2adfa-393a-4f7d-afec-55f6674aa074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:55.355256 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2adfa-393a-4f7d-afec-55f6674aa074 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:55.357685 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2adfa-393a-4f7d-afec-55f6674aa074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:55.361887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2adfa-393a-4f7d-afec-55f6674aa074 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:55.457808 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1026/2054] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:47:55 2022] POST /identity/v3/users => generated 284 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:47:55.461763 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f296597-c3c5-411d-93de-24182e8fb918 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:55.470631 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1029/2055] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:47:55 2022] POST /identity/v3/users => generated 288 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:55.480671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2adfa-393a-4f7d-afec-55f6674aa074 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:55.482767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f296597-c3c5-411d-93de-24182e8fb918 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:55.493714 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08754cb-10a0-46fe-ac19-2872128f5727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:55.494789 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08754cb-10a0-46fe-ac19-2872128f5727 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:55.495247 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08754cb-10a0-46fe-ac19-2872128f5727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:55.495846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08754cb-10a0-46fe-ac19-2872128f5727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:55.497564 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08754cb-10a0-46fe-ac19-2872128f5727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:55.497683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2adfa-393a-4f7d-afec-55f6674aa074 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:55.502751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08754cb-10a0-46fe-ac19-2872128f5727 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:55.506207 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47373920-bb42-4cf2-a1be-cc34a8b4732a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:55.507500 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47373920-bb42-4cf2-a1be-cc34a8b4732a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:55.508138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47373920-bb42-4cf2-a1be-cc34a8b4732a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:55.508701 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47373920-bb42-4cf2-a1be-cc34a8b4732a admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:55.510351 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47373920-bb42-4cf2-a1be-cc34a8b4732a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:55.514833 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47373920-bb42-4cf2-a1be-cc34a8b4732a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:55.520510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1027/2056] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:47:55.526549 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08754cb-10a0-46fe-ac19-2872128f5727 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:55.539929 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1030/2057] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:55.542944 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47373920-bb42-4cf2-a1be-cc34a8b4732a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:55.543743 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08754cb-10a0-46fe-ac19-2872128f5727 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:55.554209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00776858-7b5b-49c9-9e87-82971ca8fe7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:55.555562 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00776858-7b5b-49c9-9e87-82971ca8fe7d admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:55.556876 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00776858-7b5b-49c9-9e87-82971ca8fe7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:55.557416 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00776858-7b5b-49c9-9e87-82971ca8fe7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:55.558763 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00776858-7b5b-49c9-9e87-82971ca8fe7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:55.562655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47373920-bb42-4cf2-a1be-cc34a8b4732a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:55.562891 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00776858-7b5b-49c9-9e87-82971ca8fe7d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:55.571324 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7832bea-e4d3-44bc-968c-7213e2777c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:55.572557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7832bea-e4d3-44bc-968c-7213e2777c1e admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:55.573303 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7832bea-e4d3-44bc-968c-7213e2777c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:55.573760 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7832bea-e4d3-44bc-968c-7213e2777c1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:55.576376 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7832bea-e4d3-44bc-968c-7213e2777c1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:55.579052 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7832bea-e4d3-44bc-968c-7213e2777c1e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:55.581017 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1028/2058] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:55.588057 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:55.589364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00776858-7b5b-49c9-9e87-82971ca8fe7d admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:55.592940 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-00776858-7b5b-49c9-9e87-82971ca8fe7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:55.601693 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1031/2059] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:47:55.601693 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7832bea-e4d3-44bc-968c-7213e2777c1e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:55.611240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00776858-7b5b-49c9-9e87-82971ca8fe7d admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:55.617520 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ea569-77de-4121-b96c-5dec651002a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:55.618421 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ea569-77de-4121-b96c-5dec651002a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:55.618711 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ea569-77de-4121-b96c-5dec651002a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:55.619014 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ea569-77de-4121-b96c-5dec651002a9 admin admin] PATH_INFO: `/v3/projects/48ac5e93576143899d5906ac93229e24` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:55.620462 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ea569-77de-4121-b96c-5dec651002a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48ac5e93576143899d5906ac93229e24)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:55.622565 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ea569-77de-4121-b96c-5dec651002a9 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:55.631988 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7832bea-e4d3-44bc-968c-7213e2777c1e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:55.653949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb6346-37f2-4fd9-80af-38f6cdf0fdcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:55.654651 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb6346-37f2-4fd9-80af-38f6cdf0fdcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:55.655010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb6346-37f2-4fd9-80af-38f6cdf0fdcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:55.655479 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb6346-37f2-4fd9-80af-38f6cdf0fdcb admin admin] PATH_INFO: `/v3/projects/239c7041fd9b460cb7582dbebe806e99/users/98c4429caaaf4ede8a44d86c3b1a46cd/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:55.684441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb6346-37f2-4fd9-80af-38f6cdf0fdcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=239c7041fd9b460cb7582dbebe806e99, user_id=98c4429caaaf4ede8a44d86c3b1a46cd, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:55.689614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb6346-37f2-4fd9-80af-38f6cdf0fdcb admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:55.760335 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1032/2060] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:55 2022] PUT /identity/v3/projects/239c7041fd9b460cb7582dbebe806e99/users/98c4429caaaf4ede8a44d86c3b1a46cd/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:55.761575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb6346-37f2-4fd9-80af-38f6cdf0fdcb admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:55.776296 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-200ea569-77de-4121-b96c-5dec651002a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48ac5e93576143899d5906ac93229e24', 'request_id': 'req-200ea569-77de-4121-b96c-5dec651002a9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:55.784430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-200ea569-77de-4121-b96c-5dec651002a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48ac5e93576143899d5906ac93229e24', 'request_id': 'req-200ea569-77de-4121-b96c-5dec651002a9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:55.792889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-200ea569-77de-4121-b96c-5dec651002a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48ac5e93576143899d5906ac93229e24', 'request_id': 'req-200ea569-77de-4121-b96c-5dec651002a9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:55.847407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb6346-37f2-4fd9-80af-38f6cdf0fdcb admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:55.866628 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-200ea569-77de-4121-b96c-5dec651002a9 admin admin] The token cache is being invalidate because project 48ac5e93576143899d5906ac93229e24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:47:55.867076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-200ea569-77de-4121-b96c-5dec651002a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:55.868029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-200ea569-77de-4121-b96c-5dec651002a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:47:55.869888 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1029/2061] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:47:55 2022] DELETE /identity/v3/projects/48ac5e93576143899d5906ac93229e24 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:55.872356 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14e39d-0a2e-41de-bce9-8cbd35a06cad None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:55.873088 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14e39d-0a2e-41de-bce9-8cbd35a06cad None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:55.875250 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14e39d-0a2e-41de-bce9-8cbd35a06cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:55.918425 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-7e14e39d-0a2e-41de-bce9-8cbd35a06cad None None] MFA Rules not processed for user `98c4429caaaf4ede8a44d86c3b1a46cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:55.920408 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7e14e39d-0a2e-41de-bce9-8cbd35a06cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:55.938683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2a1e7-7e27-430a-993b-70d3241a933f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:55.940623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2a1e7-7e27-430a-993b-70d3241a933f admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:55.941090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2a1e7-7e27-430a-993b-70d3241a933f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:55.941548 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2a1e7-7e27-430a-993b-70d3241a933f admin admin] PATH_INFO: `/v3/projects/c1db22b0e1734b1ea27a68ccdb9338cb/users/a161ff7fe3e141de9e5acec2778f84fa/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:55.976351 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f2a1e7-7e27-430a-993b-70d3241a933f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1db22b0e1734b1ea27a68ccdb9338cb, user_id=a161ff7fe3e141de9e5acec2778f84fa, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:55.980541 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f2a1e7-7e27-430a-993b-70d3241a933f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:56.035962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1030/2062] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:55 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:56.038393 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eded66f-fe99-4181-9e33-c754558d2c90 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:56.042290 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eded66f-fe99-4181-9e33-c754558d2c90 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:56.042290 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eded66f-fe99-4181-9e33-c754558d2c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:56.067363 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-3eded66f-fe99-4181-9e33-c754558d2c90 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:56.069400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3eded66f-fe99-4181-9e33-c754558d2c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:56.079719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1033/2063] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:55 2022] PUT /identity/v3/projects/c1db22b0e1734b1ea27a68ccdb9338cb/users/a161ff7fe3e141de9e5acec2778f84fa/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:56.082083 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2a1e7-7e27-430a-993b-70d3241a933f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:56.085154 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-34f2a1e7-7e27-430a-993b-70d3241a933f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:56.107639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:56.160840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:56.235325 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1031/2064] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:56.235325 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd555f4-f79b-4525-bcf0-0314b53c7a4b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:56.235325 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd555f4-f79b-4525-bcf0-0314b53c7a4b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:56.235993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2a1e7-7e27-430a-993b-70d3241a933f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:56.236087 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd555f4-f79b-4525-bcf0-0314b53c7a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:56.244412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821c1d5-4999-4424-9bbe-dd8326d7bdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:56.245698 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821c1d5-4999-4424-9bbe-dd8326d7bdb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:56.246249 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821c1d5-4999-4424-9bbe-dd8326d7bdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:56.248472 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821c1d5-4999-4424-9bbe-dd8326d7bdb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:56.249493 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821c1d5-4999-4424-9bbe-dd8326d7bdb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:56.253211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821c1d5-4999-4424-9bbe-dd8326d7bdb2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:56.285952 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-9cd555f4-f79b-4525-bcf0-0314b53c7a4b None None] MFA Rules not processed for user `a161ff7fe3e141de9e5acec2778f84fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:56.287823 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9cd555f4-f79b-4525-bcf0-0314b53c7a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:56.368018 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1032/2065] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:56.369665 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8f78b-47f4-4fb4-8a4c-35d018834da1 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:56.369995 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8f78b-47f4-4fb4-8a4c-35d018834da1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:56.370239 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8f78b-47f4-4fb4-8a4c-35d018834da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:56.396006 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-02d8f78b-47f4-4fb4-8a4c-35d018834da1 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:56.397307 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-02d8f78b-47f4-4fb4-8a4c-35d018834da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:56.405825 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1033/2066] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:56.406681 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8f78b-47f4-4fb4-8a4c-35d018834da1 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:56.410340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-02d8f78b-47f4-4fb4-8a4c-35d018834da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:56.422292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1034/2067] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:47:56 2022] POST /identity/v3/projects => generated 367 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:56.422292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821c1d5-4999-4424-9bbe-dd8326d7bdb2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:56.520800 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8f78b-47f4-4fb4-8a4c-35d018834da1 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:56.526771 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821c1d5-4999-4424-9bbe-dd8326d7bdb2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:56.533997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8b102-ce47-496c-8dbf-880e009eef33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:56.534851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8b102-ce47-496c-8dbf-880e009eef33 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:56.535244 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8b102-ce47-496c-8dbf-880e009eef33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:56.535615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8b102-ce47-496c-8dbf-880e009eef33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:56.536694 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8b102-ce47-496c-8dbf-880e009eef33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:47:56.537063 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8b102-ce47-496c-8dbf-880e009eef33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:56.540184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8b102-ce47-496c-8dbf-880e009eef33 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:56.560786 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1035/2068] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:47:56.562879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8b102-ce47-496c-8dbf-880e009eef33 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:56.582095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8b102-ce47-496c-8dbf-880e009eef33 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:56.591384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28991b3d-f317-4155-b542-a7555e649c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:56.592664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28991b3d-f317-4155-b542-a7555e649c6a admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:56.593164 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28991b3d-f317-4155-b542-a7555e649c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:56.593650 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28991b3d-f317-4155-b542-a7555e649c6a admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:56.594136 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7155cb6-7427-4b62-b53e-9a68b2c578fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:56.594864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7155cb6-7427-4b62-b53e-9a68b2c578fa service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:56.594974 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28991b3d-f317-4155-b542-a7555e649c6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:56.595251 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7155cb6-7427-4b62-b53e-9a68b2c578fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:56.595587 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7155cb6-7427-4b62-b53e-9a68b2c578fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:56.598911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28991b3d-f317-4155-b542-a7555e649c6a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:56.638433 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:56.670957 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7155cb6-7427-4b62-b53e-9a68b2c578fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:56.672738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7155cb6-7427-4b62-b53e-9a68b2c578fa service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:56.686162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1034/2069] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:47:56.687616 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7155cb6-7427-4b62-b53e-9a68b2c578fa service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:56.705858 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7155cb6-7427-4b62-b53e-9a68b2c578fa service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:56.711951 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bc409-9a0f-421b-8aa5-20854cdcd331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:56.712769 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bc409-9a0f-421b-8aa5-20854cdcd331 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:56.713216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bc409-9a0f-421b-8aa5-20854cdcd331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:56.713651 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bc409-9a0f-421b-8aa5-20854cdcd331 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:56.714410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897bc409-9a0f-421b-8aa5-20854cdcd331 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:56.716693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897bc409-9a0f-421b-8aa5-20854cdcd331 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:56.734458 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1036/2070] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:47:56 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:56.740286 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28991b3d-f317-4155-b542-a7555e649c6a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:56.751938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28991b3d-f317-4155-b542-a7555e649c6a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:56.757255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5d42e-756d-4030-a2c8-36622a678004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:56.757886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5d42e-756d-4030-a2c8-36622a678004 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:56.758168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5d42e-756d-4030-a2c8-36622a678004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:56.758450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5d42e-756d-4030-a2c8-36622a678004 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:56.759198 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5d42e-756d-4030-a2c8-36622a678004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:56.760996 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5d42e-756d-4030-a2c8-36622a678004 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:56.775829 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1037/2071] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:47:56.787399 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5d42e-756d-4030-a2c8-36622a678004 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:56.808070 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5d42e-756d-4030-a2c8-36622a678004 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:56.813821 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4e550-c7f6-4b58-8b07-8cfd45d4bfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:56.814723 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4e550-c7f6-4b58-8b07-8cfd45d4bfb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:56.815132 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4e550-c7f6-4b58-8b07-8cfd45d4bfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:56.815452 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4e550-c7f6-4b58-8b07-8cfd45d4bfb6 admin admin] PATH_INFO: `/v3/projects/31b310acda534c90ae007c0d9bac54b0/users/847f2defa29842dc8b9d72fa7ff9e5f8/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:56.834639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4e550-c7f6-4b58-8b07-8cfd45d4bfb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31b310acda534c90ae007c0d9bac54b0, user_id=847f2defa29842dc8b9d72fa7ff9e5f8, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:56.838199 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4e550-c7f6-4b58-8b07-8cfd45d4bfb6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:56.878707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1035/2072] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:47:56 2022] POST /identity/v3/projects => generated 391 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:47:56.880679 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bc409-9a0f-421b-8aa5-20854cdcd331 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:56.969081 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bc409-9a0f-421b-8aa5-20854cdcd331 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:56.974376 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fcd7b1-4fb8-4291-b495-a3a4dba4f492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:56.975059 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcd7b1-4fb8-4291-b495-a3a4dba4f492 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:56.975343 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcd7b1-4fb8-4291-b495-a3a4dba4f492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:56.975618 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcd7b1-4fb8-4291-b495-a3a4dba4f492 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:56.976371 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fcd7b1-4fb8-4291-b495-a3a4dba4f492 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:56.978369 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fcd7b1-4fb8-4291-b495-a3a4dba4f492 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:57.016453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1038/2073] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:56 2022] PUT /identity/v3/projects/31b310acda534c90ae007c0d9bac54b0/users/847f2defa29842dc8b9d72fa7ff9e5f8/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:57.021236 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4e550-c7f6-4b58-8b07-8cfd45d4bfb6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:57.066707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:57.094686 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4e550-c7f6-4b58-8b07-8cfd45d4bfb6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:57.099612 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf532a-3a34-4cbe-a947-9543e59f5e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:57.100336 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf532a-3a34-4cbe-a947-9543e59f5e23 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:57.100618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf532a-3a34-4cbe-a947-9543e59f5e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:57.100913 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf532a-3a34-4cbe-a947-9543e59f5e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:57.101652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf532a-3a34-4cbe-a947-9543e59f5e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:57.103294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf532a-3a34-4cbe-a947-9543e59f5e23 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:57.122944 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1039/2074] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:47:57.127592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf532a-3a34-4cbe-a947-9543e59f5e23 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:57.139984 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf532a-3a34-4cbe-a947-9543e59f5e23 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:57.149315 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1036/2075] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:47:56 2022] POST /identity/v3/users => generated 296 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:47:57.149599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32176b24-c982-4212-8ad6-e8a3cb614ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:57.149599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32176b24-c982-4212-8ad6-e8a3cb614ae9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:57.149599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32176b24-c982-4212-8ad6-e8a3cb614ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:57.149599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32176b24-c982-4212-8ad6-e8a3cb614ae9 admin admin] PATH_INFO: `/v3/projects/31b310acda534c90ae007c0d9bac54b0/users/847f2defa29842dc8b9d72fa7ff9e5f8/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:57.150252 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32176b24-c982-4212-8ad6-e8a3cb614ae9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31b310acda534c90ae007c0d9bac54b0, user_id=847f2defa29842dc8b9d72fa7ff9e5f8, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:57.151381 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fcd7b1-4fb8-4291-b495-a3a4dba4f492 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:57.154661 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32176b24-c982-4212-8ad6-e8a3cb614ae9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:57.169807 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1040/2076] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:57 2022] PUT /identity/v3/projects/31b310acda534c90ae007c0d9bac54b0/users/847f2defa29842dc8b9d72fa7ff9e5f8/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:47:57.170027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fcd7b1-4fb8-4291-b495-a3a4dba4f492 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:57.188447 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7be6ad-30ee-4019-ae09-c683ccb521e9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:57.188986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7be6ad-30ee-4019-ae09-c683ccb521e9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:57.189482 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7be6ad-30ee-4019-ae09-c683ccb521e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:57.241019 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ec7be6ad-30ee-4019-ae09-c683ccb521e9 None None] MFA Rules not processed for user `847f2defa29842dc8b9d72fa7ff9e5f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:57.243477 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ec7be6ad-30ee-4019-ae09-c683ccb521e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:57.257031 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1931dd-78e3-4140-be18-f0d429f76716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:57.257730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1931dd-78e3-4140-be18-f0d429f76716 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:57.258022 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1931dd-78e3-4140-be18-f0d429f76716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:57.258300 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1931dd-78e3-4140-be18-f0d429f76716 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:57.259113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1931dd-78e3-4140-be18-f0d429f76716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:57.261096 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1931dd-78e3-4140-be18-f0d429f76716 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:57.274939 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1037/2077] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:47:57.280234 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1931dd-78e3-4140-be18-f0d429f76716 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:57.292198 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1931dd-78e3-4140-be18-f0d429f76716 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:57.300703 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38c899-eef8-4243-9539-d697b870e76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:57.301647 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38c899-eef8-4243-9539-d697b870e76d admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:57.301960 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38c899-eef8-4243-9539-d697b870e76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:57.302204 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38c899-eef8-4243-9539-d697b870e76d admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:57.302960 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38c899-eef8-4243-9539-d697b870e76d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:57.306295 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38c899-eef8-4243-9539-d697b870e76d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:57.320966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1038/2078] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:47:57.325092 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38c899-eef8-4243-9539-d697b870e76d admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:57.337090 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38c899-eef8-4243-9539-d697b870e76d admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:57.344906 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd309a4d-627b-47ac-8984-b85486e184e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:57.347647 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd309a4d-627b-47ac-8984-b85486e184e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:57.348380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd309a4d-627b-47ac-8984-b85486e184e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:57.349014 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd309a4d-627b-47ac-8984-b85486e184e2 admin admin] PATH_INFO: `/v3/projects/8ee917172e884491b686fbc5d18311ae/users/3d0c4544a19747aa9678812e11ac26ab/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:57.376564 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1041/2079] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:57 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:47:57.378557 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd309a4d-627b-47ac-8984-b85486e184e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ee917172e884491b686fbc5d18311ae, user_id=3d0c4544a19747aa9678812e11ac26ab, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:57.382286 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd309a4d-627b-47ac-8984-b85486e184e2 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:57.389987 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7be6ad-30ee-4019-ae09-c683ccb521e9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:57.391569 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ec7be6ad-30ee-4019-ae09-c683ccb521e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:57.466280 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7be6ad-30ee-4019-ae09-c683ccb521e9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:57.470243 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303abbf4-0461-417d-855d-ae4e09c83a10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:57.470919 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303abbf4-0461-417d-855d-ae4e09c83a10 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:57.471205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303abbf4-0461-417d-855d-ae4e09c83a10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:57.471482 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303abbf4-0461-417d-855d-ae4e09c83a10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:57.478139 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1039/2080] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:47:57 2022] PUT /identity/v3/projects/8ee917172e884491b686fbc5d18311ae/users/3d0c4544a19747aa9678812e11ac26ab/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:47:57.484753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303abbf4-0461-417d-855d-ae4e09c83a10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:57.486227 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303abbf4-0461-417d-855d-ae4e09c83a10 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:57.489976 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce3872-8d1a-4bcc-aee5-6e534f8e4d84 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:57.490239 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce3872-8d1a-4bcc-aee5-6e534f8e4d84 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:57.490524 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce3872-8d1a-4bcc-aee5-6e534f8e4d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:57.498150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1042/2081] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:57 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:47:57.499343 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:57.525856 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-13ce3872-8d1a-4bcc-aee5-6e534f8e4d84 None None] MFA Rules not processed for user `3d0c4544a19747aa9678812e11ac26ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:47:57.527116 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-13ce3872-8d1a-4bcc-aee5-6e534f8e4d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:57.603532 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1040/2082] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:47:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:47:57.645706 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303abbf4-0461-417d-855d-ae4e09c83a10 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:57.655466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303abbf4-0461-417d-855d-ae4e09c83a10 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:57.713847 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:47:57.720855 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5aa96-02e5-4143-8a67-5625e4fdffcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:57.721556 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5aa96-02e5-4143-8a67-5625e4fdffcf service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:57.721842 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5aa96-02e5-4143-8a67-5625e4fdffcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:57.722126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5aa96-02e5-4143-8a67-5625e4fdffcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:57.733082 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f5aa96-02e5-4143-8a67-5625e4fdffcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:57.734565 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f5aa96-02e5-4143-8a67-5625e4fdffcf service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:57.745877 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1043/2083] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:47:57.978269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce3872-8d1a-4bcc-aee5-6e534f8e4d84 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:57.991621 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce3872-8d1a-4bcc-aee5-6e534f8e4d84 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:57.995612 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e409ff-1ef6-46ec-9f5b-1bf56a7eaf4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:57.996317 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e409ff-1ef6-46ec-9f5b-1bf56a7eaf4a service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:57.996623 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e409ff-1ef6-46ec-9f5b-1bf56a7eaf4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:58.002996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e409ff-1ef6-46ec-9f5b-1bf56a7eaf4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:58.063012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e409ff-1ef6-46ec-9f5b-1bf56a7eaf4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:58.064703 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e409ff-1ef6-46ec-9f5b-1bf56a7eaf4a service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:47:58.075027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1041/2084] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:47:57 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:47:59.645288 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5aa96-02e5-4143-8a67-5625e4fdffcf service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:47:59.648382 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a4f5aa96-02e5-4143-8a67-5625e4fdffcf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:47:59.737821 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5aa96-02e5-4143-8a67-5625e4fdffcf service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:47:59.744623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b3fc4-90b6-4f1d-814b-3d45b09bad3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:47:59.745648 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b3fc4-90b6-4f1d-814b-3d45b09bad3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:47:59.746056 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b3fc4-90b6-4f1d-814b-3d45b09bad3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:47:59.746511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b3fc4-90b6-4f1d-814b-3d45b09bad3d admin admin] PATH_INFO: `/v3/users/98dd498f93394a8388c95b3a1fbccf10` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:47:59.749180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b3fc4-90b6-4f1d-814b-3d45b09bad3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=98dd498f93394a8388c95b3a1fbccf10)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:47:59.752047 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b3fc4-90b6-4f1d-814b-3d45b09bad3d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:00.014507 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f76b3fc4-90b6-4f1d-814b-3d45b09bad3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98dd498f93394a8388c95b3a1fbccf10'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:00.015205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f76b3fc4-90b6-4f1d-814b-3d45b09bad3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98dd498f93394a8388c95b3a1fbccf10'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:00.015923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f76b3fc4-90b6-4f1d-814b-3d45b09bad3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98dd498f93394a8388c95b3a1fbccf10'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:00.030976 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f76b3fc4-90b6-4f1d-814b-3d45b09bad3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98dd498f93394a8388c95b3a1fbccf10'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:00.042511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-f76b3fc4-90b6-4f1d-814b-3d45b09bad3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98dd498f93394a8388c95b3a1fbccf10'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:00.170790 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1044/2085] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:47:59 2022] DELETE /identity/v3/users/98dd498f93394a8388c95b3a1fbccf10 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:00.175524 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372e9d1-50b4-4020-8ede-a076ee0080d8 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:00.176094 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372e9d1-50b4-4020-8ede-a076ee0080d8 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:00.176194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372e9d1-50b4-4020-8ede-a076ee0080d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:00.194358 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c372e9d1-50b4-4020-8ede-a076ee0080d8 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:00.195566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c372e9d1-50b4-4020-8ede-a076ee0080d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:00.277275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1042/2086] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:00.288213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b3fc4-90b6-4f1d-814b-3d45b09bad3d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:00.289961 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f76b3fc4-90b6-4f1d-814b-3d45b09bad3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:00.396633 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b3fc4-90b6-4f1d-814b-3d45b09bad3d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:00.402406 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441ee64-9587-4b21-b66b-fc2a95443330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:00.403754 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441ee64-9587-4b21-b66b-fc2a95443330 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:00.404127 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441ee64-9587-4b21-b66b-fc2a95443330 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:00.404604 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441ee64-9587-4b21-b66b-fc2a95443330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:00.425246 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441ee64-9587-4b21-b66b-fc2a95443330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:00.426974 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441ee64-9587-4b21-b66b-fc2a95443330 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:00.442318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1045/2087] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:48:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:48:01.102898 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372e9d1-50b4-4020-8ede-a076ee0080d8 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:01.105447 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c372e9d1-50b4-4020-8ede-a076ee0080d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:01.121691 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372e9d1-50b4-4020-8ede-a076ee0080d8 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:01.126906 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33e0b8-c317-4683-a892-49d84d2b61b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:01.127883 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33e0b8-c317-4683-a892-49d84d2b61b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:01.128164 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33e0b8-c317-4683-a892-49d84d2b61b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:01.128431 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33e0b8-c317-4683-a892-49d84d2b61b7 admin admin] PATH_INFO: `/v3/projects/77750133d6024906b47e2d3d8c854dc2` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:01.129701 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33e0b8-c317-4683-a892-49d84d2b61b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77750133d6024906b47e2d3d8c854dc2)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:01.131638 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33e0b8-c317-4683-a892-49d84d2b61b7 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:01.273917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:01.292551 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4a33e0b8-c317-4683-a892-49d84d2b61b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77750133d6024906b47e2d3d8c854dc2', 'request_id': 'req-4a33e0b8-c317-4683-a892-49d84d2b61b7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:01.298207 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4a33e0b8-c317-4683-a892-49d84d2b61b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77750133d6024906b47e2d3d8c854dc2', 'request_id': 'req-4a33e0b8-c317-4683-a892-49d84d2b61b7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:01.304761 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4a33e0b8-c317-4683-a892-49d84d2b61b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77750133d6024906b47e2d3d8c854dc2', 'request_id': 'req-4a33e0b8-c317-4683-a892-49d84d2b61b7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:01.433174 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4a33e0b8-c317-4683-a892-49d84d2b61b7 admin admin] The token cache is being invalidate because project 77750133d6024906b47e2d3d8c854dc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:48:01.437374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4a33e0b8-c317-4683-a892-49d84d2b61b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:01.437374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4a33e0b8-c317-4683-a892-49d84d2b61b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:01.437374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1043/2088] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:48:01 2022] DELETE /identity/v3/projects/77750133d6024906b47e2d3d8c854dc2 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:01.443298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441ee64-9587-4b21-b66b-fc2a95443330 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:01.461162 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441ee64-9587-4b21-b66b-fc2a95443330 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:01.476843 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:01.533387 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34218b4-e7cf-4186-b575-d47d4240ed07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:01.535551 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34218b4-e7cf-4186-b575-d47d4240ed07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:01.535551 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34218b4-e7cf-4186-b575-d47d4240ed07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:01.535551 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34218b4-e7cf-4186-b575-d47d4240ed07 admin admin] PATH_INFO: `/v3/users/9ab5dfdb90794e86ab07feb630343bb1` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:01.536169 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34218b4-e7cf-4186-b575-d47d4240ed07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab5dfdb90794e86ab07feb630343bb1)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:01.538134 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34218b4-e7cf-4186-b575-d47d4240ed07 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:01.859009 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c34218b4-e7cf-4186-b575-d47d4240ed07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab5dfdb90794e86ab07feb630343bb1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:01.860141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c34218b4-e7cf-4186-b575-d47d4240ed07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab5dfdb90794e86ab07feb630343bb1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:01.860871 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c34218b4-e7cf-4186-b575-d47d4240ed07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab5dfdb90794e86ab07feb630343bb1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:01.876594 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c34218b4-e7cf-4186-b575-d47d4240ed07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab5dfdb90794e86ab07feb630343bb1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:01.885620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-c34218b4-e7cf-4186-b575-d47d4240ed07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab5dfdb90794e86ab07feb630343bb1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:01.965068 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1046/2089] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:48:01 2022] DELETE /identity/v3/users/9ab5dfdb90794e86ab07feb630343bb1 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:02.493570 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33e0b8-c317-4683-a892-49d84d2b61b7 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:02.494750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4a33e0b8-c317-4683-a892-49d84d2b61b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:02.511217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33e0b8-c317-4683-a892-49d84d2b61b7 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:02.579330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534485c-c608-4c2d-9df3-7dbcafdb39d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:02.580353 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534485c-c608-4c2d-9df3-7dbcafdb39d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:02.580625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534485c-c608-4c2d-9df3-7dbcafdb39d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:02.580898 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534485c-c608-4c2d-9df3-7dbcafdb39d9 admin admin] PATH_INFO: `/v3/projects/6bdffeef468f49d4bf618d12d3c68389` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:02.582125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534485c-c608-4c2d-9df3-7dbcafdb39d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bdffeef468f49d4bf618d12d3c68389)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:02.584071 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534485c-c608-4c2d-9df3-7dbcafdb39d9 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:02.724037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8534485c-c608-4c2d-9df3-7dbcafdb39d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bdffeef468f49d4bf618d12d3c68389', 'request_id': 'req-8534485c-c608-4c2d-9df3-7dbcafdb39d9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:02.730032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8534485c-c608-4c2d-9df3-7dbcafdb39d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bdffeef468f49d4bf618d12d3c68389', 'request_id': 'req-8534485c-c608-4c2d-9df3-7dbcafdb39d9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:02.737815 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8534485c-c608-4c2d-9df3-7dbcafdb39d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bdffeef468f49d4bf618d12d3c68389', 'request_id': 'req-8534485c-c608-4c2d-9df3-7dbcafdb39d9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:02.826455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8534485c-c608-4c2d-9df3-7dbcafdb39d9 admin admin] The token cache is being invalidate because project 6bdffeef468f49d4bf618d12d3c68389 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:48:02.826455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8534485c-c608-4c2d-9df3-7dbcafdb39d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:02.826455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8534485c-c608-4c2d-9df3-7dbcafdb39d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:02.826455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1044/2090] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:48:02 2022] DELETE /identity/v3/projects/6bdffeef468f49d4bf618d12d3c68389 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:02.842773 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc224e62-33d8-47da-89da-1859d63b1b42 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:02.843276 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc224e62-33d8-47da-89da-1859d63b1b42 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:02.843481 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc224e62-33d8-47da-89da-1859d63b1b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:02.882290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-bc224e62-33d8-47da-89da-1859d63b1b42 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:02.883574 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bc224e62-33d8-47da-89da-1859d63b1b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:02.958209 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1047/2091] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:02.963216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbfae9-14c6-41df-a2dd-f460b65c603c None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:02.963510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbfae9-14c6-41df-a2dd-f460b65c603c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:02.963838 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbfae9-14c6-41df-a2dd-f460b65c603c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:02.984001 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-e9fbfae9-14c6-41df-a2dd-f460b65c603c None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:02.985589 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e9fbfae9-14c6-41df-a2dd-f460b65c603c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:02.994400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1045/2092] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:03.000315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc224e62-33d8-47da-89da-1859d63b1b42 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:03.011272 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc224e62-33d8-47da-89da-1859d63b1b42 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:03.017268 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82136e4-da80-4c16-b8f2-339c84372c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:03.017949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82136e4-da80-4c16-b8f2-339c84372c85 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:03.018234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82136e4-da80-4c16-b8f2-339c84372c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:03.018514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82136e4-da80-4c16-b8f2-339c84372c85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:03.019263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82136e4-da80-4c16-b8f2-339c84372c85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:48:03.019572 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82136e4-da80-4c16-b8f2-339c84372c85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:03.021249 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82136e4-da80-4c16-b8f2-339c84372c85 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:03.037939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1048/2093] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:48:03.042686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fbfae9-14c6-41df-a2dd-f460b65c603c None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:03.053088 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fbfae9-14c6-41df-a2dd-f460b65c603c None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:03.059376 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4db6ad-84de-440b-8851-b9c0d2dc0acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:03.061666 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4db6ad-84de-440b-8851-b9c0d2dc0acf admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:03.061666 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4db6ad-84de-440b-8851-b9c0d2dc0acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:03.061666 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4db6ad-84de-440b-8851-b9c0d2dc0acf admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:03.061666 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4db6ad-84de-440b-8851-b9c0d2dc0acf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:03.063338 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4db6ad-84de-440b-8851-b9c0d2dc0acf admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:03.212177 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1046/2094] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:48:03 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:48:03.213326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82136e4-da80-4c16-b8f2-339c84372c85 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:03.224275 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82136e4-da80-4c16-b8f2-339c84372c85 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:03.237966 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:03.294364 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d86d7f-c7ee-49af-9677-c36bab77201f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:03.295357 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d86d7f-c7ee-49af-9677-c36bab77201f admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:03.295868 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d86d7f-c7ee-49af-9677-c36bab77201f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:03.296383 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d86d7f-c7ee-49af-9677-c36bab77201f admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:03.297278 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d86d7f-c7ee-49af-9677-c36bab77201f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:03.299658 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d86d7f-c7ee-49af-9677-c36bab77201f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:03.411100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1049/2095] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:48:03 2022] POST /identity/v3/users => generated 294 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:48:03.416819 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4db6ad-84de-440b-8851-b9c0d2dc0acf admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:03.428801 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4db6ad-84de-440b-8851-b9c0d2dc0acf admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:03.433971 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a3572-a37b-4f4b-915a-92111b959b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:03.435534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a3572-a37b-4f4b-915a-92111b959b14 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:03.435972 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a3572-a37b-4f4b-915a-92111b959b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:03.436255 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a3572-a37b-4f4b-915a-92111b959b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:03.437496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a3572-a37b-4f4b-915a-92111b959b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:03.438720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a3572-a37b-4f4b-915a-92111b959b14 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:03.450523 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1047/2096] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:48:03.455177 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d86d7f-c7ee-49af-9677-c36bab77201f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:03.466430 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d86d7f-c7ee-49af-9677-c36bab77201f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:03.471431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049bfd8-5c47-4bf9-bb56-f5bd6a657227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:03.472330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049bfd8-5c47-4bf9-bb56-f5bd6a657227 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:03.472622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049bfd8-5c47-4bf9-bb56-f5bd6a657227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:03.472906 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049bfd8-5c47-4bf9-bb56-f5bd6a657227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:03.473696 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049bfd8-5c47-4bf9-bb56-f5bd6a657227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:03.475347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049bfd8-5c47-4bf9-bb56-f5bd6a657227 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:03.486777 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1050/2097] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:48:03.491273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a3572-a37b-4f4b-915a-92111b959b14 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:03.498880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:03.502724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a3572-a37b-4f4b-915a-92111b959b14 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:03.507334 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c587857-7e6d-4184-a75b-2cd70484eee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:03.508272 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c587857-7e6d-4184-a75b-2cd70484eee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:03.508549 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c587857-7e6d-4184-a75b-2cd70484eee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:03.508822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c587857-7e6d-4184-a75b-2cd70484eee1 admin admin] PATH_INFO: `/v3/projects/15d92bea797743859dd7c452809c4756/users/f03c26f5c73440a684848c5836f081c3/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:03.524521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c587857-7e6d-4184-a75b-2cd70484eee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15d92bea797743859dd7c452809c4756, user_id=f03c26f5c73440a684848c5836f081c3, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:03.527196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c587857-7e6d-4184-a75b-2cd70484eee1 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:03.638952 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1048/2098] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:48:03 2022] PUT /identity/v3/projects/15d92bea797743859dd7c452809c4756/users/f03c26f5c73440a684848c5836f081c3/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:03.650438 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05586c9f-6862-49d6-84af-88b17fdc70a3 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:03.650777 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05586c9f-6862-49d6-84af-88b17fdc70a3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:03.651070 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05586c9f-6862-49d6-84af-88b17fdc70a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:03.679877 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-05586c9f-6862-49d6-84af-88b17fdc70a3 None None] MFA Rules not processed for user `f03c26f5c73440a684848c5836f081c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:03.681112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-05586c9f-6862-49d6-84af-88b17fdc70a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:03.765357 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1051/2099] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:03.770518 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c587857-7e6d-4184-a75b-2cd70484eee1 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:03.890344 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c587857-7e6d-4184-a75b-2cd70484eee1 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:03.896711 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a4a3c-e541-4f10-9ab0-659e68fc6520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:03.897388 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a4a3c-e541-4f10-9ab0-659e68fc6520 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:03.898307 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a4a3c-e541-4f10-9ab0-659e68fc6520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:03.898307 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a4a3c-e541-4f10-9ab0-659e68fc6520 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:03.898771 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a4a3c-e541-4f10-9ab0-659e68fc6520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:03.900835 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a4a3c-e541-4f10-9ab0-659e68fc6520 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:04.010712 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1049/2100] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:48:03 2022] POST /identity/v3/projects => generated 387 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:48:04.012226 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05586c9f-6862-49d6-84af-88b17fdc70a3 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:04.025034 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05586c9f-6862-49d6-84af-88b17fdc70a3 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:04.097397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc77ca-357d-49f4-a078-6b33b5267154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:04.098110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc77ca-357d-49f4-a078-6b33b5267154 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:04.098391 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc77ca-357d-49f4-a078-6b33b5267154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:04.098714 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc77ca-357d-49f4-a078-6b33b5267154 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:04.099587 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc77ca-357d-49f4-a078-6b33b5267154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:04.101688 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc77ca-357d-49f4-a078-6b33b5267154 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:04.225517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1052/2101] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:48:04 2022] POST /identity/v3/users => generated 300 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:48:04.229267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a4a3c-e541-4f10-9ab0-659e68fc6520 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:04.242956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a4a3c-e541-4f10-9ab0-659e68fc6520 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:04.248976 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f68236-aef9-4a5a-a6f1-1abcb7c8ace0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:04.252900 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f68236-aef9-4a5a-a6f1-1abcb7c8ace0 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:04.252900 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f68236-aef9-4a5a-a6f1-1abcb7c8ace0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:04.252900 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f68236-aef9-4a5a-a6f1-1abcb7c8ace0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:04.252900 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f68236-aef9-4a5a-a6f1-1abcb7c8ace0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:04.252900 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f68236-aef9-4a5a-a6f1-1abcb7c8ace0 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:04.264570 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1050/2102] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:48:04.268915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc77ca-357d-49f4-a078-6b33b5267154 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:04.279217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc77ca-357d-49f4-a078-6b33b5267154 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:04.284255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffec8ab-ed76-4f93-a90c-ea204db78cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:04.285187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffec8ab-ed76-4f93-a90c-ea204db78cba admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:04.285447 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffec8ab-ed76-4f93-a90c-ea204db78cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:04.285725 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffec8ab-ed76-4f93-a90c-ea204db78cba admin admin] PATH_INFO: `/v3/projects/9529a4212cec4c23bd18d1c352b471f3/users/94041fba7d514745934be8d8504daade/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:04.300258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:04.304941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffec8ab-ed76-4f93-a90c-ea204db78cba admin admin] RBAC: Authorizing `identity:create_grant(project_id=9529a4212cec4c23bd18d1c352b471f3, user_id=94041fba7d514745934be8d8504daade, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:04.307684 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffec8ab-ed76-4f93-a90c-ea204db78cba admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:04.433876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1053/2103] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:48:04 2022] PUT /identity/v3/projects/9529a4212cec4c23bd18d1c352b471f3/users/94041fba7d514745934be8d8504daade/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:48:04.445934 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f68236-aef9-4a5a-a6f1-1abcb7c8ace0 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:04.465773 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f68236-aef9-4a5a-a6f1-1abcb7c8ace0 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:04.537240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37519756-b64a-462f-9098-d9649a524011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:04.538097 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37519756-b64a-462f-9098-d9649a524011 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:04.538220 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37519756-b64a-462f-9098-d9649a524011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:04.538473 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37519756-b64a-462f-9098-d9649a524011 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:04.539245 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37519756-b64a-462f-9098-d9649a524011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:04.540929 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37519756-b64a-462f-9098-d9649a524011 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:04.559021 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1051/2104] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:48:04.566774 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffec8ab-ed76-4f93-a90c-ea204db78cba admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:04.583936 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffec8ab-ed76-4f93-a90c-ea204db78cba admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:04.590943 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7535fbf-596e-4644-84ef-790b4f3ea2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:04.592190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7535fbf-596e-4644-84ef-790b4f3ea2b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:04.592492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7535fbf-596e-4644-84ef-790b4f3ea2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:04.592832 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7535fbf-596e-4644-84ef-790b4f3ea2b9 admin admin] PATH_INFO: `/v3/projects/9529a4212cec4c23bd18d1c352b471f3/users/94041fba7d514745934be8d8504daade/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:04.596271 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7535fbf-596e-4644-84ef-790b4f3ea2b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9529a4212cec4c23bd18d1c352b471f3, user_id=94041fba7d514745934be8d8504daade, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:04.601581 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7535fbf-596e-4644-84ef-790b4f3ea2b9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:04.623113 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1054/2105] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:48:04 2022] PUT /identity/v3/projects/9529a4212cec4c23bd18d1c352b471f3/users/94041fba7d514745934be8d8504daade/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:04.639032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b82ce-b6cf-48d3-998d-6f5da1cfca99 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:04.639418 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b82ce-b6cf-48d3-998d-6f5da1cfca99 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:04.639803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b82ce-b6cf-48d3-998d-6f5da1cfca99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:04.685080 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-125b82ce-b6cf-48d3-998d-6f5da1cfca99 None None] MFA Rules not processed for user `94041fba7d514745934be8d8504daade`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:04.686102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-125b82ce-b6cf-48d3-998d-6f5da1cfca99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:04.745927 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:04.787884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1052/2106] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:48:04.800754 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7535fbf-596e-4644-84ef-790b4f3ea2b9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:04.802952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a7535fbf-596e-4644-84ef-790b4f3ea2b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:04.883107 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7535fbf-596e-4644-84ef-790b4f3ea2b9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:04.889842 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df9ed4-833e-4009-bdf5-a616db20aeff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:04.890990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df9ed4-833e-4009-bdf5-a616db20aeff service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:04.891491 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df9ed4-833e-4009-bdf5-a616db20aeff service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:04.892222 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df9ed4-833e-4009-bdf5-a616db20aeff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:04.910929 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df9ed4-833e-4009-bdf5-a616db20aeff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:04.914043 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df9ed4-833e-4009-bdf5-a616db20aeff service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:04.931285 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1055/2107] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:48:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:48:05.920498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b82ce-b6cf-48d3-998d-6f5da1cfca99 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:05.922927 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-125b82ce-b6cf-48d3-998d-6f5da1cfca99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:06.072122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b82ce-b6cf-48d3-998d-6f5da1cfca99 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:06.077741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665efadc-c249-4445-a090-cd400ebff342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:06.079039 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665efadc-c249-4445-a090-cd400ebff342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:06.079412 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665efadc-c249-4445-a090-cd400ebff342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:06.079889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665efadc-c249-4445-a090-cd400ebff342 admin admin] PATH_INFO: `/v3/users/a161ff7fe3e141de9e5acec2778f84fa` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:06.081660 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665efadc-c249-4445-a090-cd400ebff342 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a161ff7fe3e141de9e5acec2778f84fa)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:06.084967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665efadc-c249-4445-a090-cd400ebff342 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:06.389984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-665efadc-c249-4445-a090-cd400ebff342 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a161ff7fe3e141de9e5acec2778f84fa'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:06.390685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-665efadc-c249-4445-a090-cd400ebff342 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a161ff7fe3e141de9e5acec2778f84fa'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:06.391444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-665efadc-c249-4445-a090-cd400ebff342 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a161ff7fe3e141de9e5acec2778f84fa'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:06.409385 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-665efadc-c249-4445-a090-cd400ebff342 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a161ff7fe3e141de9e5acec2778f84fa'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:06.417112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-665efadc-c249-4445-a090-cd400ebff342 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a161ff7fe3e141de9e5acec2778f84fa'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:06.540958 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1053/2108] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:48:05 2022] DELETE /identity/v3/users/a161ff7fe3e141de9e5acec2778f84fa => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:06.546115 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a21108-f384-4df9-aeb0-618723b87073 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:06.546623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a21108-f384-4df9-aeb0-618723b87073 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:06.546756 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a21108-f384-4df9-aeb0-618723b87073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:06.565492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-48a21108-f384-4df9-aeb0-618723b87073 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:06.566692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-48a21108-f384-4df9-aeb0-618723b87073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:06.638106 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1056/2109] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:06.638387 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665efadc-c249-4445-a090-cd400ebff342 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:06.639908 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-665efadc-c249-4445-a090-cd400ebff342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:06.650190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a21108-f384-4df9-aeb0-618723b87073 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:06.651904 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-48a21108-f384-4df9-aeb0-618723b87073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:06.655505 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665efadc-c249-4445-a090-cd400ebff342 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:06.666259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b02c2e-c13e-4f31-8501-06d095e6ca48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:06.666259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b02c2e-c13e-4f31-8501-06d095e6ca48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:06.666259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b02c2e-c13e-4f31-8501-06d095e6ca48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:06.666259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b02c2e-c13e-4f31-8501-06d095e6ca48 admin admin] PATH_INFO: `/v3/users/98c4429caaaf4ede8a44d86c3b1a46cd` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:06.666684 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b02c2e-c13e-4f31-8501-06d095e6ca48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98c4429caaaf4ede8a44d86c3b1a46cd)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:06.666684 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b02c2e-c13e-4f31-8501-06d095e6ca48 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:06.742655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a21108-f384-4df9-aeb0-618723b87073 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:06.746850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c7038-b1ca-4dca-a84a-0215df36848b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:06.748103 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c7038-b1ca-4dca-a84a-0215df36848b service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:06.748483 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c7038-b1ca-4dca-a84a-0215df36848b service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:06.748832 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c7038-b1ca-4dca-a84a-0215df36848b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:06.759189 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c7038-b1ca-4dca-a84a-0215df36848b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:06.760831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c7038-b1ca-4dca-a84a-0215df36848b service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:06.772050 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1057/2110] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:48:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:48:07.026066 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e7b02c2e-c13e-4f31-8501-06d095e6ca48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98c4429caaaf4ede8a44d86c3b1a46cd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:07.027493 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e7b02c2e-c13e-4f31-8501-06d095e6ca48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98c4429caaaf4ede8a44d86c3b1a46cd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:07.031940 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e7b02c2e-c13e-4f31-8501-06d095e6ca48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98c4429caaaf4ede8a44d86c3b1a46cd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:07.034776 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:07.066953 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e7b02c2e-c13e-4f31-8501-06d095e6ca48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98c4429caaaf4ede8a44d86c3b1a46cd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:07.082220 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e7b02c2e-c13e-4f31-8501-06d095e6ca48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98c4429caaaf4ede8a44d86c3b1a46cd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:07.226605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1054/2111] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:48:06 2022] DELETE /identity/v3/users/98c4429caaaf4ede8a44d86c3b1a46cd => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:48:07.233838 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c58ec-3bb0-4652-a6c5-500da6108d7b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:07.233838 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c58ec-3bb0-4652-a6c5-500da6108d7b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:07.234324 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c58ec-3bb0-4652-a6c5-500da6108d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:07.266939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-794c58ec-3bb0-4652-a6c5-500da6108d7b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:07.279244 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-794c58ec-3bb0-4652-a6c5-500da6108d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:07.296320 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:07.300381 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b02c2e-c13e-4f31-8501-06d095e6ca48 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:07.301951 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e7b02c2e-c13e-4f31-8501-06d095e6ca48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:07.418006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1058/2112] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:48:07.419973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c58ec-3bb0-4652-a6c5-500da6108d7b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:07.422196 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-794c58ec-3bb0-4652-a6c5-500da6108d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:07.444151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c58ec-3bb0-4652-a6c5-500da6108d7b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:07.456284 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c56f8-fb90-4c12-9c26-22f6dae64823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:07.457732 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c56f8-fb90-4c12-9c26-22f6dae64823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:07.458295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c56f8-fb90-4c12-9c26-22f6dae64823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:07.460627 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c56f8-fb90-4c12-9c26-22f6dae64823 admin admin] PATH_INFO: `/v3/projects/c1db22b0e1734b1ea27a68ccdb9338cb` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:07.461091 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c56f8-fb90-4c12-9c26-22f6dae64823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1db22b0e1734b1ea27a68ccdb9338cb)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:07.464882 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c56f8-fb90-4c12-9c26-22f6dae64823 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:07.486370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b02c2e-c13e-4f31-8501-06d095e6ca48 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:07.495054 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f6bfd-b27a-4f0d-891d-6efe5b248f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:07.497101 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f6bfd-b27a-4f0d-891d-6efe5b248f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:07.497591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f6bfd-b27a-4f0d-891d-6efe5b248f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:07.498027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f6bfd-b27a-4f0d-891d-6efe5b248f90 admin admin] PATH_INFO: `/v3/users/f03c26f5c73440a684848c5836f081c3` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:07.504998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f6bfd-b27a-4f0d-891d-6efe5b248f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f03c26f5c73440a684848c5836f081c3)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:07.507337 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f6bfd-b27a-4f0d-891d-6efe5b248f90 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:07.633040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-126c56f8-fb90-4c12-9c26-22f6dae64823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1db22b0e1734b1ea27a68ccdb9338cb', 'request_id': 'req-126c56f8-fb90-4c12-9c26-22f6dae64823'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:07.641092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-126c56f8-fb90-4c12-9c26-22f6dae64823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1db22b0e1734b1ea27a68ccdb9338cb', 'request_id': 'req-126c56f8-fb90-4c12-9c26-22f6dae64823'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:07.649341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-126c56f8-fb90-4c12-9c26-22f6dae64823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1db22b0e1734b1ea27a68ccdb9338cb', 'request_id': 'req-126c56f8-fb90-4c12-9c26-22f6dae64823'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:07.750787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-126c56f8-fb90-4c12-9c26-22f6dae64823 admin admin] The token cache is being invalidate because project c1db22b0e1734b1ea27a68ccdb9338cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:48:07.751394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-126c56f8-fb90-4c12-9c26-22f6dae64823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:07.752519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-126c56f8-fb90-4c12-9c26-22f6dae64823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:07.755143 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1059/2113] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:48:07 2022] DELETE /identity/v3/projects/c1db22b0e1734b1ea27a68ccdb9338cb => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:07.756532 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c56f8-fb90-4c12-9c26-22f6dae64823 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:07.759528 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-126c56f8-fb90-4c12-9c26-22f6dae64823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:07.765324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c92f6bfd-b27a-4f0d-891d-6efe5b248f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f03c26f5c73440a684848c5836f081c3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:07.766240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c92f6bfd-b27a-4f0d-891d-6efe5b248f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f03c26f5c73440a684848c5836f081c3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:07.767177 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c92f6bfd-b27a-4f0d-891d-6efe5b248f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f03c26f5c73440a684848c5836f081c3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:07.791835 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c92f6bfd-b27a-4f0d-891d-6efe5b248f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f03c26f5c73440a684848c5836f081c3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:07.805824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c92f6bfd-b27a-4f0d-891d-6efe5b248f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f03c26f5c73440a684848c5836f081c3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:07.843082 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c56f8-fb90-4c12-9c26-22f6dae64823 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:07.848979 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c0d85-0f5f-4959-9de0-945f392d2cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:07.861141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c0d85-0f5f-4959-9de0-945f392d2cfb service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:07.861141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c0d85-0f5f-4959-9de0-945f392d2cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:07.861141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c0d85-0f5f-4959-9de0-945f392d2cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:07.861141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-372c0d85-0f5f-4959-9de0-945f392d2cfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:07.978080 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1055/2114] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:48:07 2022] DELETE /identity/v3/users/f03c26f5c73440a684848c5836f081c3 => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:07.980673 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1039f-ed3f-44f5-87a2-23f96eb027c8 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:07.981822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1039f-ed3f-44f5-87a2-23f96eb027c8 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:07.982280 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1039f-ed3f-44f5-87a2-23f96eb027c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:08.024526 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c0d85-0f5f-4959-9de0-945f392d2cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:08.029571 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c0d85-0f5f-4959-9de0-945f392d2cfb service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:08.041902 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:08.056100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d0f1039f-ed3f-44f5-87a2-23f96eb027c8 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:08.057056 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1060/2115] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:48:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:48:08.058411 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d0f1039f-ed3f-44f5-87a2-23f96eb027c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:08.061173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8ba34-2dfe-4702-849b-3b30756dfd53 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:08.061608 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8ba34-2dfe-4702-849b-3b30756dfd53 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:08.062370 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8ba34-2dfe-4702-849b-3b30756dfd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:08.107477 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-1ab8ba34-2dfe-4702-849b-3b30756dfd53 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:08.109755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1ab8ba34-2dfe-4702-849b-3b30756dfd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:08.218369 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:08.249730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1056/2116] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:08.257900 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9821d54-065d-412a-b598-3d311faf40fe None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:08.258230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9821d54-065d-412a-b598-3d311faf40fe None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:08.258478 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9821d54-065d-412a-b598-3d311faf40fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:08.260447 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1061/2117] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:08.273634 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8ba34-2dfe-4702-849b-3b30756dfd53 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:08.275224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-1ab8ba34-2dfe-4702-849b-3b30756dfd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:08.281918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d9821d54-065d-412a-b598-3d311faf40fe None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:08.283165 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d9821d54-065d-412a-b598-3d311faf40fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:08.292699 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1057/2118] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:48:08.297789 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9821d54-065d-412a-b598-3d311faf40fe None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:08.316015 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9821d54-065d-412a-b598-3d311faf40fe None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:08.325741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adae421-4d0b-42d7-8358-6a84d2f34a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:08.327049 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adae421-4d0b-42d7-8358-6a84d2f34a90 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:08.328307 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adae421-4d0b-42d7-8358-6a84d2f34a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:08.328762 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adae421-4d0b-42d7-8358-6a84d2f34a90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:08.330556 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adae421-4d0b-42d7-8358-6a84d2f34a90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:48:08.331455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adae421-4d0b-42d7-8358-6a84d2f34a90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:08.336375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adae421-4d0b-42d7-8358-6a84d2f34a90 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:08.361346 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8ba34-2dfe-4702-849b-3b30756dfd53 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:08.365712 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32ae21-2d75-47a8-8081-b1b00d56be04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:08.365926 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1058/2119] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:48:08.366565 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32ae21-2d75-47a8-8081-b1b00d56be04 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:08.366941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32ae21-2d75-47a8-8081-b1b00d56be04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:08.367386 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32ae21-2d75-47a8-8081-b1b00d56be04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:08.380607 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adae421-4d0b-42d7-8358-6a84d2f34a90 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:08.383050 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32ae21-2d75-47a8-8081-b1b00d56be04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:08.385168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32ae21-2d75-47a8-8081-b1b00d56be04 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:08.399144 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1062/2120] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:48:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:48:08.402510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adae421-4d0b-42d7-8358-6a84d2f34a90 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:08.415285 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb4071-763a-4a54-86e1-d48f03c716bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:08.416210 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb4071-763a-4a54-86e1-d48f03c716bb admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:08.420495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb4071-763a-4a54-86e1-d48f03c716bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:08.420741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb4071-763a-4a54-86e1-d48f03c716bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:08.423958 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb4071-763a-4a54-86e1-d48f03c716bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:08.430754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb4071-763a-4a54-86e1-d48f03c716bb admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:08.633994 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1059/2121] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:48:08 2022] POST /identity/v3/projects => generated 365 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:48:08.638623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32ae21-2d75-47a8-8081-b1b00d56be04 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:08.648564 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32ae21-2d75-47a8-8081-b1b00d56be04 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:08.717631 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1a749-1812-4c8b-8b6b-52716dfc3eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:08.718525 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1a749-1812-4c8b-8b6b-52716dfc3eac admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:08.718903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1a749-1812-4c8b-8b6b-52716dfc3eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:08.719382 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1a749-1812-4c8b-8b6b-52716dfc3eac admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:08.720300 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1a749-1812-4c8b-8b6b-52716dfc3eac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:08.722755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1a749-1812-4c8b-8b6b-52716dfc3eac admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:08.750772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb4071-763a-4a54-86e1-d48f03c716bb admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:08.753276 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-42fb4071-763a-4a54-86e1-d48f03c716bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:08.777414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb4071-763a-4a54-86e1-d48f03c716bb admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:08.795888 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebb9c0-8b97-4c99-a15d-dc5ce8b910ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:08.797064 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebb9c0-8b97-4c99-a15d-dc5ce8b910ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:08.797362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebb9c0-8b97-4c99-a15d-dc5ce8b910ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:08.797671 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebb9c0-8b97-4c99-a15d-dc5ce8b910ca admin admin] PATH_INFO: `/v3/projects/239c7041fd9b460cb7582dbebe806e99` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:08.800276 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bebb9c0-8b97-4c99-a15d-dc5ce8b910ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=239c7041fd9b460cb7582dbebe806e99)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:08.802617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bebb9c0-8b97-4c99-a15d-dc5ce8b910ca admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:08.838919 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1063/2122] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:48:08 2022] POST /identity/v3/users => generated 283 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:48:08.842395 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1a749-1812-4c8b-8b6b-52716dfc3eac admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:08.867337 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1a749-1812-4c8b-8b6b-52716dfc3eac admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:08.891627 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e43d2-857b-4021-bd31-c6014e9c90fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:08.896125 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e43d2-857b-4021-bd31-c6014e9c90fe admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:08.897037 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e43d2-857b-4021-bd31-c6014e9c90fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:08.897037 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e43d2-857b-4021-bd31-c6014e9c90fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:08.897701 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e43d2-857b-4021-bd31-c6014e9c90fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:08.900090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e43d2-857b-4021-bd31-c6014e9c90fe admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:08.912785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1064/2123] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:48:08.916480 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e43d2-857b-4021-bd31-c6014e9c90fe admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:08.931247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e43d2-857b-4021-bd31-c6014e9c90fe admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:08.940066 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee91b98-6319-47a3-868c-d9cf028d5008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:08.941630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee91b98-6319-47a3-868c-d9cf028d5008 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:08.942205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee91b98-6319-47a3-868c-d9cf028d5008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:08.942646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee91b98-6319-47a3-868c-d9cf028d5008 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:08.943996 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee91b98-6319-47a3-868c-d9cf028d5008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:08.947146 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee91b98-6319-47a3-868c-d9cf028d5008 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:08.969450 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1065/2124] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:48:08.975391 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee91b98-6319-47a3-868c-d9cf028d5008 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:08.988255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee91b98-6319-47a3-868c-d9cf028d5008 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:08.994120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b32c62-339f-4249-b634-ffb844ea4195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:08.995113 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b32c62-339f-4249-b634-ffb844ea4195 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:08.995486 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b32c62-339f-4249-b634-ffb844ea4195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:08.995814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b32c62-339f-4249-b634-ffb844ea4195 admin admin] PATH_INFO: `/v3/projects/1cbac3a1b7dc4042afaaebb6ad02614d/users/1964f2aa98b7478ab181bcaaedc39204/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:09.001647 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:09.037379 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b32c62-339f-4249-b634-ffb844ea4195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cbac3a1b7dc4042afaaebb6ad02614d, user_id=1964f2aa98b7478ab181bcaaedc39204, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:09.040271 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b32c62-339f-4249-b634-ffb844ea4195 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:09.080038 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2bebb9c0-8b97-4c99-a15d-dc5ce8b910ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '239c7041fd9b460cb7582dbebe806e99', 'request_id': 'req-2bebb9c0-8b97-4c99-a15d-dc5ce8b910ca'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:09.092352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2bebb9c0-8b97-4c99-a15d-dc5ce8b910ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '239c7041fd9b460cb7582dbebe806e99', 'request_id': 'req-2bebb9c0-8b97-4c99-a15d-dc5ce8b910ca'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:09.100305 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2bebb9c0-8b97-4c99-a15d-dc5ce8b910ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '239c7041fd9b460cb7582dbebe806e99', 'request_id': 'req-2bebb9c0-8b97-4c99-a15d-dc5ce8b910ca'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:09.142752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1066/2125] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:48:08 2022] PUT /identity/v3/projects/1cbac3a1b7dc4042afaaebb6ad02614d/users/1964f2aa98b7478ab181bcaaedc39204/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:48:09.143787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b32c62-339f-4249-b634-ffb844ea4195 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:09.145972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-82b32c62-339f-4249-b634-ffb844ea4195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:09.236713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b32c62-339f-4249-b634-ffb844ea4195 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:09.241972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ae512-748e-4318-ac7f-5f749cf6a081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:09.242858 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ae512-748e-4318-ac7f-5f749cf6a081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:09.243144 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ae512-748e-4318-ac7f-5f749cf6a081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:09.243425 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ae512-748e-4318-ac7f-5f749cf6a081 admin admin] PATH_INFO: `/v3/projects/15d92bea797743859dd7c452809c4756` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:09.244775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ae512-748e-4318-ac7f-5f749cf6a081 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15d92bea797743859dd7c452809c4756)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:09.247237 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ae512-748e-4318-ac7f-5f749cf6a081 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:09.284720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2bebb9c0-8b97-4c99-a15d-dc5ce8b910ca admin admin] The token cache is being invalidate because project 239c7041fd9b460cb7582dbebe806e99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:48:09.285064 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2bebb9c0-8b97-4c99-a15d-dc5ce8b910ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:09.285857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2bebb9c0-8b97-4c99-a15d-dc5ce8b910ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:09.287645 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1060/2126] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:48:08 2022] DELETE /identity/v3/projects/239c7041fd9b460cb7582dbebe806e99 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:48:09.289755 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237443f-d210-47c4-b4c8-085029cc5394 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:09.290135 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237443f-d210-47c4-b4c8-085029cc5394 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:09.290520 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237443f-d210-47c4-b4c8-085029cc5394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:09.323792 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-3237443f-d210-47c4-b4c8-085029cc5394 None None] MFA Rules not processed for user `1964f2aa98b7478ab181bcaaedc39204`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:09.326067 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3237443f-d210-47c4-b4c8-085029cc5394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:09.395211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-324ae512-748e-4318-ac7f-5f749cf6a081 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15d92bea797743859dd7c452809c4756', 'request_id': 'req-324ae512-748e-4318-ac7f-5f749cf6a081'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:09.401374 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-324ae512-748e-4318-ac7f-5f749cf6a081 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15d92bea797743859dd7c452809c4756', 'request_id': 'req-324ae512-748e-4318-ac7f-5f749cf6a081'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:09.409383 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1061/2127] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:09.410571 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-324ae512-748e-4318-ac7f-5f749cf6a081 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15d92bea797743859dd7c452809c4756', 'request_id': 'req-324ae512-748e-4318-ac7f-5f749cf6a081'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:09.410741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237443f-d210-47c4-b4c8-085029cc5394 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:09.412834 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3237443f-d210-47c4-b4c8-085029cc5394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:09.429203 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237443f-d210-47c4-b4c8-085029cc5394 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:09.443775 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:09.494732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600d42d-640f-4198-a576-284dfe35c4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:09.495640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600d42d-640f-4198-a576-284dfe35c4b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:09.495953 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600d42d-640f-4198-a576-284dfe35c4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:09.496237 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600d42d-640f-4198-a576-284dfe35c4b4 admin admin] PATH_INFO: `/v3/users/847f2defa29842dc8b9d72fa7ff9e5f8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:09.497689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600d42d-640f-4198-a576-284dfe35c4b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=847f2defa29842dc8b9d72fa7ff9e5f8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:09.499861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600d42d-640f-4198-a576-284dfe35c4b4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:09.501971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-324ae512-748e-4318-ac7f-5f749cf6a081 admin admin] The token cache is being invalidate because project 15d92bea797743859dd7c452809c4756 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:48:09.502268 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-324ae512-748e-4318-ac7f-5f749cf6a081 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:09.509454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-324ae512-748e-4318-ac7f-5f749cf6a081 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:09.509454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1067/2128] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:48:09 2022] DELETE /identity/v3/projects/15d92bea797743859dd7c452809c4756 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:09.509454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ae512-748e-4318-ac7f-5f749cf6a081 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:09.509454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-324ae512-748e-4318-ac7f-5f749cf6a081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:09.571748 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ae512-748e-4318-ac7f-5f749cf6a081 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:09.576868 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7bb63-b6cd-4f37-a756-ff617e6744f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:09.577603 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7bb63-b6cd-4f37-a756-ff617e6744f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:09.578152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7bb63-b6cd-4f37-a756-ff617e6744f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:09.578569 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7bb63-b6cd-4f37-a756-ff617e6744f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:09.581163 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e2a7bb63-b6cd-4f37-a756-ff617e6744f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:09.666950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7bb63-b6cd-4f37-a756-ff617e6744f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:09.669067 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7bb63-b6cd-4f37-a756-ff617e6744f6 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:09.679603 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1068/2129] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:48:09 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:48:09.680543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7bb63-b6cd-4f37-a756-ff617e6744f6 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:09.682037 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e2a7bb63-b6cd-4f37-a756-ff617e6744f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:09.707611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7bb63-b6cd-4f37-a756-ff617e6744f6 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:09.716139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519be57c-e511-460e-842f-6c0dd9a1eb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:09.717602 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519be57c-e511-460e-842f-6c0dd9a1eb78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:09.718072 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519be57c-e511-460e-842f-6c0dd9a1eb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:09.718530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519be57c-e511-460e-842f-6c0dd9a1eb78 admin admin] PATH_INFO: `/v3/users/94041fba7d514745934be8d8504daade` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:09.720728 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519be57c-e511-460e-842f-6c0dd9a1eb78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94041fba7d514745934be8d8504daade)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:09.724616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519be57c-e511-460e-842f-6c0dd9a1eb78 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:09.769488 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:09.788595 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9600d42d-640f-4198-a576-284dfe35c4b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '847f2defa29842dc8b9d72fa7ff9e5f8'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:09.789334 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9600d42d-640f-4198-a576-284dfe35c4b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '847f2defa29842dc8b9d72fa7ff9e5f8'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:09.790232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9600d42d-640f-4198-a576-284dfe35c4b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '847f2defa29842dc8b9d72fa7ff9e5f8'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:09.809578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9600d42d-640f-4198-a576-284dfe35c4b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '847f2defa29842dc8b9d72fa7ff9e5f8'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:09.818679 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9600d42d-640f-4198-a576-284dfe35c4b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '847f2defa29842dc8b9d72fa7ff9e5f8'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:09.982626 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1062/2130] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:48:09 2022] DELETE /identity/v3/users/847f2defa29842dc8b9d72fa7ff9e5f8 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:10.042817 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-519be57c-e511-460e-842f-6c0dd9a1eb78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94041fba7d514745934be8d8504daade'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:10.043600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-519be57c-e511-460e-842f-6c0dd9a1eb78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94041fba7d514745934be8d8504daade'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:10.048489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-519be57c-e511-460e-842f-6c0dd9a1eb78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94041fba7d514745934be8d8504daade'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:10.066142 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-519be57c-e511-460e-842f-6c0dd9a1eb78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94041fba7d514745934be8d8504daade'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:10.074310 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-519be57c-e511-460e-842f-6c0dd9a1eb78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94041fba7d514745934be8d8504daade'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:10.210336 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1069/2131] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:48:09 2022] DELETE /identity/v3/users/94041fba7d514745934be8d8504daade => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:10.610834 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600d42d-640f-4198-a576-284dfe35c4b4 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:10.613935 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9600d42d-640f-4198-a576-284dfe35c4b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:10.751609 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600d42d-640f-4198-a576-284dfe35c4b4 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:10.863717 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839988e9-e7e1-4fbb-ab99-450f7758cfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:10.863717 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839988e9-e7e1-4fbb-ab99-450f7758cfb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:10.863717 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839988e9-e7e1-4fbb-ab99-450f7758cfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:10.863717 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839988e9-e7e1-4fbb-ab99-450f7758cfb2 admin admin] PATH_INFO: `/v3/projects/31b310acda534c90ae007c0d9bac54b0` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:10.865463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519be57c-e511-460e-842f-6c0dd9a1eb78 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:10.867414 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-519be57c-e511-460e-842f-6c0dd9a1eb78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:10.868729 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839988e9-e7e1-4fbb-ab99-450f7758cfb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31b310acda534c90ae007c0d9bac54b0)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:10.873462 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839988e9-e7e1-4fbb-ab99-450f7758cfb2 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:10.905586 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519be57c-e511-460e-842f-6c0dd9a1eb78 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:10.933443 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb1abba-9ad0-4950-9336-5389e968a350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:10.934901 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1abba-9ad0-4950-9336-5389e968a350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:10.946770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1abba-9ad0-4950-9336-5389e968a350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:10.946770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1abba-9ad0-4950-9336-5389e968a350 admin admin] PATH_INFO: `/v3/projects/9529a4212cec4c23bd18d1c352b471f3` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:10.946770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb1abba-9ad0-4950-9336-5389e968a350 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9529a4212cec4c23bd18d1c352b471f3)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:10.946770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb1abba-9ad0-4950-9336-5389e968a350 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:11.066145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-839988e9-e7e1-4fbb-ab99-450f7758cfb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31b310acda534c90ae007c0d9bac54b0', 'request_id': 'req-839988e9-e7e1-4fbb-ab99-450f7758cfb2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:11.072665 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-839988e9-e7e1-4fbb-ab99-450f7758cfb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31b310acda534c90ae007c0d9bac54b0', 'request_id': 'req-839988e9-e7e1-4fbb-ab99-450f7758cfb2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:11.080645 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-839988e9-e7e1-4fbb-ab99-450f7758cfb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31b310acda534c90ae007c0d9bac54b0', 'request_id': 'req-839988e9-e7e1-4fbb-ab99-450f7758cfb2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:11.086725 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:11.119087 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bcb1abba-9ad0-4950-9336-5389e968a350 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9529a4212cec4c23bd18d1c352b471f3', 'request_id': 'req-bcb1abba-9ad0-4950-9336-5389e968a350'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:11.126838 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bcb1abba-9ad0-4950-9336-5389e968a350 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9529a4212cec4c23bd18d1c352b471f3', 'request_id': 'req-bcb1abba-9ad0-4950-9336-5389e968a350'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:11.134869 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bcb1abba-9ad0-4950-9336-5389e968a350 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9529a4212cec4c23bd18d1c352b471f3', 'request_id': 'req-bcb1abba-9ad0-4950-9336-5389e968a350'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:11.177886 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-839988e9-e7e1-4fbb-ab99-450f7758cfb2 admin admin] The token cache is being invalidate because project 31b310acda534c90ae007c0d9bac54b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:48:11.178235 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-839988e9-e7e1-4fbb-ab99-450f7758cfb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:11.179022 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-839988e9-e7e1-4fbb-ab99-450f7758cfb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:11.180757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1063/2132] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:48:10 2022] DELETE /identity/v3/projects/31b310acda534c90ae007c0d9bac54b0 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:11.187611 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec87239-0e60-473c-bf04-6a52b84bcc63 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:11.190076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec87239-0e60-473c-bf04-6a52b84bcc63 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:11.190076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec87239-0e60-473c-bf04-6a52b84bcc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:11.208629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-fec87239-0e60-473c-bf04-6a52b84bcc63 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:11.210193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-fec87239-0e60-473c-bf04-6a52b84bcc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:11.243983 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bcb1abba-9ad0-4950-9336-5389e968a350 admin admin] The token cache is being invalidate because project 9529a4212cec4c23bd18d1c352b471f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:48:11.244325 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bcb1abba-9ad0-4950-9336-5389e968a350 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:11.245380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bcb1abba-9ad0-4950-9336-5389e968a350 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:11.247151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1070/2133] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:48:10 2022] DELETE /identity/v3/projects/9529a4212cec4c23bd18d1c352b471f3 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:11.253978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff06322-c0bc-4e7f-9419-84e4f0d5ac2c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:11.254263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff06322-c0bc-4e7f-9419-84e4f0d5ac2c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:11.254537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff06322-c0bc-4e7f-9419-84e4f0d5ac2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:11.275952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-cff06322-c0bc-4e7f-9419-84e4f0d5ac2c None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:11.278114 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cff06322-c0bc-4e7f-9419-84e4f0d5ac2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:11.299335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1064/2134] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:48:11.301737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25e32d-66a3-4b71-b29c-09df75347397 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:11.302220 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25e32d-66a3-4b71-b29c-09df75347397 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:11.302568 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25e32d-66a3-4b71-b29c-09df75347397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:11.326894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-4a25e32d-66a3-4b71-b29c-09df75347397 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:11.328491 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4a25e32d-66a3-4b71-b29c-09df75347397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:11.340094 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1065/2135] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:48:11.348560 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25e32d-66a3-4b71-b29c-09df75347397 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:11.360306 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25e32d-66a3-4b71-b29c-09df75347397 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:11.366292 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fdf5a-1939-4e82-b3d8-c3a67ceadd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:11.366976 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fdf5a-1939-4e82-b3d8-c3a67ceadd16 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:11.367497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fdf5a-1939-4e82-b3d8-c3a67ceadd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:11.367815 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fdf5a-1939-4e82-b3d8-c3a67ceadd16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:11.376953 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fdf5a-1939-4e82-b3d8-c3a67ceadd16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:48:11.376953 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fdf5a-1939-4e82-b3d8-c3a67ceadd16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:11.377694 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:11.377850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fdf5a-1939-4e82-b3d8-c3a67ceadd16 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:11.392663 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1071/2136] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:11.392939 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1066/2137] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:48:11.395285 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fdf5a-1939-4e82-b3d8-c3a67ceadd16 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:11.396598 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38ddf6-22b5-485c-ba5a-f2027897f5ea None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:11.397091 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38ddf6-22b5-485c-ba5a-f2027897f5ea None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:11.397620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38ddf6-22b5-485c-ba5a-f2027897f5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:11.408787 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fdf5a-1939-4e82-b3d8-c3a67ceadd16 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:11.416008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69128ed4-4773-41d8-aad5-f412e69d8d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:11.416747 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69128ed4-4773-41d8-aad5-f412e69d8d0a admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:11.417163 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69128ed4-4773-41d8-aad5-f412e69d8d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:11.417632 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69128ed4-4773-41d8-aad5-f412e69d8d0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:11.418953 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69128ed4-4773-41d8-aad5-f412e69d8d0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:11.422039 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69128ed4-4773-41d8-aad5-f412e69d8d0a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:11.433715 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-da38ddf6-22b5-485c-ba5a-f2027897f5ea None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:11.439713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-da38ddf6-22b5-485c-ba5a-f2027897f5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:11.461112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1072/2138] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:11.466240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da38ddf6-22b5-485c-ba5a-f2027897f5ea None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:11.476996 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da38ddf6-22b5-485c-ba5a-f2027897f5ea None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:11.482708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f664b-c31c-4b42-aa78-575c2045ff8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:11.483499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f664b-c31c-4b42-aa78-575c2045ff8d admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:11.483897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f664b-c31c-4b42-aa78-575c2045ff8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:11.484230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f664b-c31c-4b42-aa78-575c2045ff8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:11.485640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f664b-c31c-4b42-aa78-575c2045ff8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:48:11.485958 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f664b-c31c-4b42-aa78-575c2045ff8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:11.487989 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f664b-c31c-4b42-aa78-575c2045ff8d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:11.508276 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1073/2139] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:48:11.509646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f664b-c31c-4b42-aa78-575c2045ff8d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:11.523926 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f664b-c31c-4b42-aa78-575c2045ff8d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:11.532566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3bc79-7b15-4410-a4e1-e97ea0935a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:11.533813 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3bc79-7b15-4410-a4e1-e97ea0935a52 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:11.534337 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3bc79-7b15-4410-a4e1-e97ea0935a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:11.534839 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3bc79-7b15-4410-a4e1-e97ea0935a52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:11.536233 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3bc79-7b15-4410-a4e1-e97ea0935a52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:11.540084 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3bc79-7b15-4410-a4e1-e97ea0935a52 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:11.581238 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1067/2140] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:48:11 2022] POST /identity/v3/projects => generated 371 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:48:11.586930 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69128ed4-4773-41d8-aad5-f412e69d8d0a admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:11.639267 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1074/2141] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:48:11 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:48:11.643759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3bc79-7b15-4410-a4e1-e97ea0935a52 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:11.669720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69128ed4-4773-41d8-aad5-f412e69d8d0a admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:11.721684 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3bc79-7b15-4410-a4e1-e97ea0935a52 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:11.728392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6085ac-9371-4331-8080-3d741e059082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:11.729360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6085ac-9371-4331-8080-3d741e059082 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:11.729728 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6085ac-9371-4331-8080-3d741e059082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:11.730136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6085ac-9371-4331-8080-3d741e059082 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:11.731198 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6085ac-9371-4331-8080-3d741e059082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:11.734018 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6085ac-9371-4331-8080-3d741e059082 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:11.741803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f126d9b-8070-4b25-8d9a-201d34a9156f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:11.746324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f126d9b-8070-4b25-8d9a-201d34a9156f admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:11.746324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f126d9b-8070-4b25-8d9a-201d34a9156f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:11.746324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f126d9b-8070-4b25-8d9a-201d34a9156f admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:11.746324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f126d9b-8070-4b25-8d9a-201d34a9156f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:11.746324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f126d9b-8070-4b25-8d9a-201d34a9156f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:11.870299 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1075/2142] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:48:11 2022] POST /identity/v3/users => generated 286 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:48:11.873262 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6085ac-9371-4331-8080-3d741e059082 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:11.873727 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1068/2143] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:48:11 2022] POST /identity/v3/users => generated 286 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:48:11.874716 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f126d9b-8070-4b25-8d9a-201d34a9156f admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:11.885488 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f126d9b-8070-4b25-8d9a-201d34a9156f admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:11.885710 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6085ac-9371-4331-8080-3d741e059082 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:11.891539 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848679a4-b9d7-48c7-80ce-1d27be815861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:11.891737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32394945-d6d3-46f1-835c-987b6f890289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:11.892242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848679a4-b9d7-48c7-80ce-1d27be815861 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:11.892357 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32394945-d6d3-46f1-835c-987b6f890289 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:11.892529 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848679a4-b9d7-48c7-80ce-1d27be815861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:11.892795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32394945-d6d3-46f1-835c-987b6f890289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:11.892940 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848679a4-b9d7-48c7-80ce-1d27be815861 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:11.893306 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32394945-d6d3-46f1-835c-987b6f890289 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:11.893538 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848679a4-b9d7-48c7-80ce-1d27be815861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:11.894182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32394945-d6d3-46f1-835c-987b6f890289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:11.895161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848679a4-b9d7-48c7-80ce-1d27be815861 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:11.896781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32394945-d6d3-46f1-835c-987b6f890289 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:11.907567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1069/2144] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:48:11.913262 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848679a4-b9d7-48c7-80ce-1d27be815861 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:11.925718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848679a4-b9d7-48c7-80ce-1d27be815861 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:11.928244 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1076/2145] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:48:11.930214 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32394945-d6d3-46f1-835c-987b6f890289 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:11.932333 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17c836-c9fc-42e3-bb02-3e38d4933950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:11.933380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17c836-c9fc-42e3-bb02-3e38d4933950 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:11.933932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17c836-c9fc-42e3-bb02-3e38d4933950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:11.934547 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17c836-c9fc-42e3-bb02-3e38d4933950 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:11.951313 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17c836-c9fc-42e3-bb02-3e38d4933950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:11.953574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17c836-c9fc-42e3-bb02-3e38d4933950 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:11.955528 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32394945-d6d3-46f1-835c-987b6f890289 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:11.966615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e90e2a-4fa5-4ea3-abd0-6cd7924361af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:11.968988 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e90e2a-4fa5-4ea3-abd0-6cd7924361af admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:11.984895 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e90e2a-4fa5-4ea3-abd0-6cd7924361af admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:11.984895 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e90e2a-4fa5-4ea3-abd0-6cd7924361af admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:11.984895 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e90e2a-4fa5-4ea3-abd0-6cd7924361af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:11.984895 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e90e2a-4fa5-4ea3-abd0-6cd7924361af admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:11.985818 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1070/2146] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:48:11.991345 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17c836-c9fc-42e3-bb02-3e38d4933950 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:12.003306 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17c836-c9fc-42e3-bb02-3e38d4933950 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:12.009577 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1077/2147] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:48:12.013650 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fbbd87-f683-440e-a2a3-3b882eaab6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:12.013650 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbbd87-f683-440e-a2a3-3b882eaab6bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:12.013650 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbbd87-f683-440e-a2a3-3b882eaab6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:12.013650 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbbd87-f683-440e-a2a3-3b882eaab6bb admin admin] PATH_INFO: `/v3/projects/1031357e1ddd40ea9bace1f26544b5ef/users/3b61737de7cc4cce89a2cd18eba7605c/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:12.015141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e90e2a-4fa5-4ea3-abd0-6cd7924361af admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:12.025942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e90e2a-4fa5-4ea3-abd0-6cd7924361af admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:12.032689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fbbd87-f683-440e-a2a3-3b882eaab6bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1031357e1ddd40ea9bace1f26544b5ef, user_id=3b61737de7cc4cce89a2cd18eba7605c, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:12.034391 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b83a9-8148-4ea7-b674-c91bd7a86c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:12.035517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b83a9-8148-4ea7-b674-c91bd7a86c2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:12.035936 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b83a9-8148-4ea7-b674-c91bd7a86c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:12.039589 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b83a9-8148-4ea7-b674-c91bd7a86c2d admin admin] PATH_INFO: `/v3/projects/796f946d907e4b5397667bf02fb0791c/users/d842d6e4dedb4239a95fb8e74673bf41/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:12.046730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fbbd87-f683-440e-a2a3-3b882eaab6bb admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:12.067753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b83a9-8148-4ea7-b674-c91bd7a86c2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=796f946d907e4b5397667bf02fb0791c, user_id=d842d6e4dedb4239a95fb8e74673bf41, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:12.073198 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b83a9-8148-4ea7-b674-c91bd7a86c2d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:12.149991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1071/2148] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:48:11 2022] PUT /identity/v3/projects/1031357e1ddd40ea9bace1f26544b5ef/users/3b61737de7cc4cce89a2cd18eba7605c/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:12.161107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923234d8-0fcb-4cc4-bda4-9c1d8ab09c48 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:12.161402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923234d8-0fcb-4cc4-bda4-9c1d8ab09c48 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:12.161879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923234d8-0fcb-4cc4-bda4-9c1d8ab09c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:12.199233 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-923234d8-0fcb-4cc4-bda4-9c1d8ab09c48 None None] MFA Rules not processed for user `3b61737de7cc4cce89a2cd18eba7605c`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:12.201895 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-923234d8-0fcb-4cc4-bda4-9c1d8ab09c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:12.227492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:12.227996 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1078/2149] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:48:12 2022] PUT /identity/v3/projects/796f946d907e4b5397667bf02fb0791c/users/d842d6e4dedb4239a95fb8e74673bf41/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:12.250582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa6807-ee14-4bea-a2a0-a8b0bf47e535 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:12.259012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa6807-ee14-4bea-a2a0-a8b0bf47e535 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:12.259521 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa6807-ee14-4bea-a2a0-a8b0bf47e535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:12.313895 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ecaa6807-ee14-4bea-a2a0-a8b0bf47e535 None None] MFA Rules not processed for user `d842d6e4dedb4239a95fb8e74673bf41`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:12.315061 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ecaa6807-ee14-4bea-a2a0-a8b0bf47e535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:12.323374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1072/2150] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:48:12.336798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923234d8-0fcb-4cc4-bda4-9c1d8ab09c48 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:12.338228 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-923234d8-0fcb-4cc4-bda4-9c1d8ab09c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:12.343438 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:12.409972 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923234d8-0fcb-4cc4-bda4-9c1d8ab09c48 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:12.412216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:12.435326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1079/2151] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:12.448805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa6807-ee14-4bea-a2a0-a8b0bf47e535 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:12.452519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ecaa6807-ee14-4bea-a2a0-a8b0bf47e535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:12.477563 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66ce8a-de46-45fc-830b-62029d505452 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:12.480412 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66ce8a-de46-45fc-830b-62029d505452 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:12.481352 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66ce8a-de46-45fc-830b-62029d505452 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:12.481692 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66ce8a-de46-45fc-830b-62029d505452 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:12.529172 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa6807-ee14-4bea-a2a0-a8b0bf47e535 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:12.533578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39315070-5899-473e-b736-c87fe415d573 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '694088ed30b94bad9d3b16f3766252df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:12.534729 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39315070-5899-473e-b736-c87fe415d573 service glance] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:12.535012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39315070-5899-473e-b736-c87fe415d573 service glance] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:12.535278 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39315070-5899-473e-b736-c87fe415d573 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:12.553067 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39315070-5899-473e-b736-c87fe415d573 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:12.556072 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39315070-5899-473e-b736-c87fe415d573 service glance] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:12.570815 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66ce8a-de46-45fc-830b-62029d505452 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:12.572668 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66ce8a-de46-45fc-830b-62029d505452 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:12.577143 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1080/2152] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:48:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:48:12.584810 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1073/2153] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:48:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:48:12.608157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39315070-5899-473e-b736-c87fe415d573 service glance] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:12.694925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39315070-5899-473e-b736-c87fe415d573 service glance] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:12.700483 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca37cb5-2257-4931-8769-e99f43d9f924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:12.701460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca37cb5-2257-4931-8769-e99f43d9f924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:12.701824 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca37cb5-2257-4931-8769-e99f43d9f924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:12.702171 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca37cb5-2257-4931-8769-e99f43d9f924 admin admin] PATH_INFO: `/v3/users/3b61737de7cc4cce89a2cd18eba7605c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:12.703879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca37cb5-2257-4931-8769-e99f43d9f924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b61737de7cc4cce89a2cd18eba7605c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:12.705930 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca37cb5-2257-4931-8769-e99f43d9f924 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:12.941261 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dca37cb5-2257-4931-8769-e99f43d9f924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b61737de7cc4cce89a2cd18eba7605c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:12.945083 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dca37cb5-2257-4931-8769-e99f43d9f924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b61737de7cc4cce89a2cd18eba7605c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:12.945083 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dca37cb5-2257-4931-8769-e99f43d9f924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b61737de7cc4cce89a2cd18eba7605c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:12.957983 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:12.960470 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dca37cb5-2257-4931-8769-e99f43d9f924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b61737de7cc4cce89a2cd18eba7605c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:12.969055 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-dca37cb5-2257-4931-8769-e99f43d9f924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b61737de7cc4cce89a2cd18eba7605c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:13.101167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1081/2154] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:48:12 2022] DELETE /identity/v3/users/3b61737de7cc4cce89a2cd18eba7605c => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:48:13.106505 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968129a-1da8-4415-aac6-34913658a136 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:13.107050 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968129a-1da8-4415-aac6-34913658a136 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:13.107050 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968129a-1da8-4415-aac6-34913658a136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:13.128102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-3968129a-1da8-4415-aac6-34913658a136 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:13.129348 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3968129a-1da8-4415-aac6-34913658a136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:13.197506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1074/2155] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:13.209281 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca37cb5-2257-4931-8769-e99f43d9f924 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:13.210733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-dca37cb5-2257-4931-8769-e99f43d9f924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:13.273324 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca37cb5-2257-4931-8769-e99f43d9f924 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:13.277131 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce26493-4171-4c21-a0c8-8998a4913138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:13.279946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce26493-4171-4c21-a0c8-8998a4913138 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:13.280276 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce26493-4171-4c21-a0c8-8998a4913138 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:13.280568 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce26493-4171-4c21-a0c8-8998a4913138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:13.295718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce26493-4171-4c21-a0c8-8998a4913138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:13.298547 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce26493-4171-4c21-a0c8-8998a4913138 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:13.319283 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1082/2156] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:48:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:48:14.015000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968129a-1da8-4415-aac6-34913658a136 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:14.016701 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3968129a-1da8-4415-aac6-34913658a136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:14.039620 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968129a-1da8-4415-aac6-34913658a136 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:14.049688 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0e51f-7310-4a47-9439-a727ca420cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:14.051283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0e51f-7310-4a47-9439-a727ca420cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:14.051795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0e51f-7310-4a47-9439-a727ca420cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:14.052360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0e51f-7310-4a47-9439-a727ca420cd5 admin admin] PATH_INFO: `/v3/projects/1031357e1ddd40ea9bace1f26544b5ef` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:14.055832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0e51f-7310-4a47-9439-a727ca420cd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1031357e1ddd40ea9bace1f26544b5ef)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:14.058636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0e51f-7310-4a47-9439-a727ca420cd5 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:14.114577 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:14.243623 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-31c0e51f-7310-4a47-9439-a727ca420cd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1031357e1ddd40ea9bace1f26544b5ef', 'request_id': 'req-31c0e51f-7310-4a47-9439-a727ca420cd5'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:14.251069 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-31c0e51f-7310-4a47-9439-a727ca420cd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1031357e1ddd40ea9bace1f26544b5ef', 'request_id': 'req-31c0e51f-7310-4a47-9439-a727ca420cd5'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:14.262454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-31c0e51f-7310-4a47-9439-a727ca420cd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1031357e1ddd40ea9bace1f26544b5ef', 'request_id': 'req-31c0e51f-7310-4a47-9439-a727ca420cd5'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:14.355497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-31c0e51f-7310-4a47-9439-a727ca420cd5 admin admin] The token cache is being invalidate because project 1031357e1ddd40ea9bace1f26544b5ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:48:14.357111 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-31c0e51f-7310-4a47-9439-a727ca420cd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:14.358240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-31c0e51f-7310-4a47-9439-a727ca420cd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:14.360853 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1075/2157] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:48:14 2022] DELETE /identity/v3/projects/1031357e1ddd40ea9bace1f26544b5ef => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:48:14.367511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477bfbdd-b0e1-4cff-820e-f793757ffc7e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:14.368175 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477bfbdd-b0e1-4cff-820e-f793757ffc7e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:14.368175 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477bfbdd-b0e1-4cff-820e-f793757ffc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:14.388682 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-477bfbdd-b0e1-4cff-820e-f793757ffc7e None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:14.389953 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-477bfbdd-b0e1-4cff-820e-f793757ffc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:14.493263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1083/2158] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:14.508065 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64ec9c-0fe3-481a-8e34-d2eb87036a1f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:14.508584 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64ec9c-0fe3-481a-8e34-d2eb87036a1f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:14.509070 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64ec9c-0fe3-481a-8e34-d2eb87036a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:14.547588 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6a64ec9c-0fe3-481a-8e34-d2eb87036a1f None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:14.550228 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6a64ec9c-0fe3-481a-8e34-d2eb87036a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:14.567954 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1076/2159] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:14.575099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477bfbdd-b0e1-4cff-820e-f793757ffc7e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:14.594921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477bfbdd-b0e1-4cff-820e-f793757ffc7e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:14.603210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891af69-66fa-410c-b651-05db988dd75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:14.604414 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891af69-66fa-410c-b651-05db988dd75c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:14.605686 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891af69-66fa-410c-b651-05db988dd75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:14.605686 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891af69-66fa-410c-b651-05db988dd75c admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:14.607714 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891af69-66fa-410c-b651-05db988dd75c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:48:14.607938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891af69-66fa-410c-b651-05db988dd75c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:14.611048 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891af69-66fa-410c-b651-05db988dd75c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:14.643439 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1084/2160] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:48:14.645706 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64ec9c-0fe3-481a-8e34-d2eb87036a1f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:14.664424 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64ec9c-0fe3-481a-8e34-d2eb87036a1f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:14.673447 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd89a13-4111-443c-afba-35c78048bb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:14.674600 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd89a13-4111-443c-afba-35c78048bb45 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:14.675113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd89a13-4111-443c-afba-35c78048bb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:14.675616 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd89a13-4111-443c-afba-35c78048bb45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:14.677073 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd89a13-4111-443c-afba-35c78048bb45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:14.680576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd89a13-4111-443c-afba-35c78048bb45 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:14.824855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1077/2161] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:48:14 2022] POST /identity/v3/projects => generated 375 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:48:14.829552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891af69-66fa-410c-b651-05db988dd75c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:14.865272 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891af69-66fa-410c-b651-05db988dd75c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:14.985817 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999946a6-2898-425e-8e68-9b9b161856a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:14.986616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999946a6-2898-425e-8e68-9b9b161856a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:14.986854 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999946a6-2898-425e-8e68-9b9b161856a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:14.987241 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999946a6-2898-425e-8e68-9b9b161856a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:14.988173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999946a6-2898-425e-8e68-9b9b161856a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:14.998395 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999946a6-2898-425e-8e68-9b9b161856a3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:15.119621 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:15.146541 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1085/2162] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:48:14 2022] POST /identity/v3/users => generated 288 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:48:15.153798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd89a13-4111-443c-afba-35c78048bb45 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:15.186736 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd89a13-4111-443c-afba-35c78048bb45 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:15.201793 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28261ef-22f0-49c1-be71-f52fcfcaebfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:15.204742 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28261ef-22f0-49c1-be71-f52fcfcaebfe admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:15.206331 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28261ef-22f0-49c1-be71-f52fcfcaebfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:15.207736 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28261ef-22f0-49c1-be71-f52fcfcaebfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:15.210190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28261ef-22f0-49c1-be71-f52fcfcaebfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:15.216000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28261ef-22f0-49c1-be71-f52fcfcaebfe admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:15.246003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1078/2163] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:48:15.253724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999946a6-2898-425e-8e68-9b9b161856a3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:15.294362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999946a6-2898-425e-8e68-9b9b161856a3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:15.302346 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12658de-6733-4ccd-8d39-66cb0410f942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:15.304221 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12658de-6733-4ccd-8d39-66cb0410f942 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:15.321350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12658de-6733-4ccd-8d39-66cb0410f942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:15.321350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12658de-6733-4ccd-8d39-66cb0410f942 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:15.321350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12658de-6733-4ccd-8d39-66cb0410f942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:15.321350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12658de-6733-4ccd-8d39-66cb0410f942 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:15.349216 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1086/2164] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:48:15.357463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28261ef-22f0-49c1-be71-f52fcfcaebfe admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:15.373687 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28261ef-22f0-49c1-be71-f52fcfcaebfe admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:15.384453 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68415380-e580-4756-8f4e-a7e8a588dede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:15.386202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68415380-e580-4756-8f4e-a7e8a588dede admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:15.386726 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68415380-e580-4756-8f4e-a7e8a588dede admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:15.387921 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68415380-e580-4756-8f4e-a7e8a588dede admin admin] PATH_INFO: `/v3/projects/15f6694a151d4a949d48f87669af7345/users/1494a19570144bc59bd95bcd0489230f/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:15.423760 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68415380-e580-4756-8f4e-a7e8a588dede admin admin] RBAC: Authorizing `identity:create_grant(project_id=15f6694a151d4a949d48f87669af7345, user_id=1494a19570144bc59bd95bcd0489230f, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:15.426859 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68415380-e580-4756-8f4e-a7e8a588dede admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:15.530106 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1079/2165] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:48:15 2022] PUT /identity/v3/projects/15f6694a151d4a949d48f87669af7345/users/1494a19570144bc59bd95bcd0489230f/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:48:15.542347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ec245-daf5-46d9-9c12-903c56bdbfc5 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:15.543796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ec245-daf5-46d9-9c12-903c56bdbfc5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:15.543796 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ec245-daf5-46d9-9c12-903c56bdbfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:15.587175 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-100ec245-daf5-46d9-9c12-903c56bdbfc5 None None] MFA Rules not processed for user `1494a19570144bc59bd95bcd0489230f`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:15.588481 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-100ec245-daf5-46d9-9c12-903c56bdbfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:15.682495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1087/2166] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:15.688360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68415380-e580-4756-8f4e-a7e8a588dede admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:15.690245 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-68415380-e580-4756-8f4e-a7e8a588dede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:15.795368 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68415380-e580-4756-8f4e-a7e8a588dede admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:15.803015 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869507c-a9dc-4a6d-b592-c1a306a1b09f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '694088ed30b94bad9d3b16f3766252df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:15.803908 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869507c-a9dc-4a6d-b592-c1a306a1b09f service glance] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:15.807755 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869507c-a9dc-4a6d-b592-c1a306a1b09f service glance] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:15.810235 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869507c-a9dc-4a6d-b592-c1a306a1b09f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:15.824608 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869507c-a9dc-4a6d-b592-c1a306a1b09f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:15.826628 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869507c-a9dc-4a6d-b592-c1a306a1b09f service glance] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:15.838117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1080/2167] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:48:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:48:22.567533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ec245-daf5-46d9-9c12-903c56bdbfc5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:22.569508 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-100ec245-daf5-46d9-9c12-903c56bdbfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:22.663981 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ec245-daf5-46d9-9c12-903c56bdbfc5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:22.695575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eec53e-4d1e-41f4-aa86-0af550d29c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:22.697109 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eec53e-4d1e-41f4-aa86-0af550d29c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:22.697581 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eec53e-4d1e-41f4-aa86-0af550d29c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:22.698040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eec53e-4d1e-41f4-aa86-0af550d29c8c admin admin] PATH_INFO: `/v3/users/3d0c4544a19747aa9678812e11ac26ab` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:22.702254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eec53e-4d1e-41f4-aa86-0af550d29c8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d0c4544a19747aa9678812e11ac26ab)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:22.708478 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eec53e-4d1e-41f4-aa86-0af550d29c8c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:22.959564 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-89eec53e-4d1e-41f4-aa86-0af550d29c8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d0c4544a19747aa9678812e11ac26ab'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:22.960938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-89eec53e-4d1e-41f4-aa86-0af550d29c8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d0c4544a19747aa9678812e11ac26ab'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:22.962257 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-89eec53e-4d1e-41f4-aa86-0af550d29c8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d0c4544a19747aa9678812e11ac26ab'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:23.000461 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-89eec53e-4d1e-41f4-aa86-0af550d29c8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d0c4544a19747aa9678812e11ac26ab'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:23.015839 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-89eec53e-4d1e-41f4-aa86-0af550d29c8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d0c4544a19747aa9678812e11ac26ab'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:23.036382 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:23.155582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1088/2168] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:48:22 2022] DELETE /identity/v3/users/3d0c4544a19747aa9678812e11ac26ab => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:23.168158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c0587-51f7-490b-b820-4c113f680ffc None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:23.169391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c0587-51f7-490b-b820-4c113f680ffc None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:23.169829 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c0587-51f7-490b-b820-4c113f680ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:23.195820 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-cf3c0587-51f7-490b-b820-4c113f680ffc None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:23.197382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-cf3c0587-51f7-490b-b820-4c113f680ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:23.254462 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:23.287246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1081/2169] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:23.301037 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eec53e-4d1e-41f4-aa86-0af550d29c8c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:23.302767 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-89eec53e-4d1e-41f4-aa86-0af550d29c8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:23.426413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eec53e-4d1e-41f4-aa86-0af550d29c8c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:23.431230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f138b-088c-4737-927e-f0ce8a37e8b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:23.432288 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f138b-088c-4737-927e-f0ce8a37e8b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:23.435733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f138b-088c-4737-927e-f0ce8a37e8b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:23.435733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f138b-088c-4737-927e-f0ce8a37e8b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:23.452762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f138b-088c-4737-927e-f0ce8a37e8b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:23.456258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f138b-088c-4737-927e-f0ce8a37e8b1 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:23.471863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1089/2170] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:48:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:48:24.154916 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c0587-51f7-490b-b820-4c113f680ffc None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:24.159309 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-cf3c0587-51f7-490b-b820-4c113f680ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:24.230126 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c0587-51f7-490b-b820-4c113f680ffc None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:24.244445 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b608174-ca9d-45dc-baa9-f07270e45af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:24.245909 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b608174-ca9d-45dc-baa9-f07270e45af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:24.246768 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b608174-ca9d-45dc-baa9-f07270e45af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:24.247959 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b608174-ca9d-45dc-baa9-f07270e45af3 admin admin] PATH_INFO: `/v3/projects/8ee917172e884491b686fbc5d18311ae` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:24.250921 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b608174-ca9d-45dc-baa9-f07270e45af3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ee917172e884491b686fbc5d18311ae)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:24.263610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b608174-ca9d-45dc-baa9-f07270e45af3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:24.445826 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5b608174-ca9d-45dc-baa9-f07270e45af3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ee917172e884491b686fbc5d18311ae', 'request_id': 'req-5b608174-ca9d-45dc-baa9-f07270e45af3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:24.458216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5b608174-ca9d-45dc-baa9-f07270e45af3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ee917172e884491b686fbc5d18311ae', 'request_id': 'req-5b608174-ca9d-45dc-baa9-f07270e45af3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:24.472372 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5b608174-ca9d-45dc-baa9-f07270e45af3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ee917172e884491b686fbc5d18311ae', 'request_id': 'req-5b608174-ca9d-45dc-baa9-f07270e45af3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:24.570758 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5b608174-ca9d-45dc-baa9-f07270e45af3 admin admin] The token cache is being invalidate because project 8ee917172e884491b686fbc5d18311ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:48:24.571460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5b608174-ca9d-45dc-baa9-f07270e45af3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:24.572288 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5b608174-ca9d-45dc-baa9-f07270e45af3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:24.574739 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1082/2171] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:48:24 2022] DELETE /identity/v3/projects/8ee917172e884491b686fbc5d18311ae => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:26.651353 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e383f7-acf8-4bc0-83d5-38f5d8243d96 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:26.652238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e383f7-acf8-4bc0-83d5-38f5d8243d96 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:26.652448 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e383f7-acf8-4bc0-83d5-38f5d8243d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:26.681193 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a6e383f7-acf8-4bc0-83d5-38f5d8243d96 None None] MFA Rules not processed for user `4e6b788e0819483580c09902a341ea6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:26.682499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a6e383f7-acf8-4bc0-83d5-38f5d8243d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:26.757084 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1090/2172] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:34.566497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=33,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:36.748737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=53 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:41.078733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4827ce-4f78-44d0-80f6-b2264cb73769 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:41.080325 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4827ce-4f78-44d0-80f6-b2264cb73769 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:41.080773 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4827ce-4f78-44d0-80f6-b2264cb73769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:41.123072 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2a4827ce-4f78-44d0-80f6-b2264cb73769 None None] MFA Rules not processed for user `2b03f8b8c51e472bbd4cf0afc9c6b632`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:41.124046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2a4827ce-4f78-44d0-80f6-b2264cb73769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:41.243072 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1083/2173] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Jul 18 08:48:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:41.378399 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e383f7-acf8-4bc0-83d5-38f5d8243d96 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:41.382253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a6e383f7-acf8-4bc0-83d5-38f5d8243d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:41.559643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e383f7-acf8-4bc0-83d5-38f5d8243d96 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:41.568885 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d796c1-fa0f-457b-86c0-8693a5be8aaa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09a5c2e804504038817cad76544dce5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:41.570176 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d796c1-fa0f-457b-86c0-8693a5be8aaa service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:41.570628 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d796c1-fa0f-457b-86c0-8693a5be8aaa service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:41.571111 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d796c1-fa0f-457b-86c0-8693a5be8aaa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:41.601932 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d796c1-fa0f-457b-86c0-8693a5be8aaa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:41.606659 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d796c1-fa0f-457b-86c0-8693a5be8aaa service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:41.627915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1091/2174] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Jul 18 08:48:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 08:48:46.863326 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4827ce-4f78-44d0-80f6-b2264cb73769 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:46.865813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2a4827ce-4f78-44d0-80f6-b2264cb73769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:46.966570 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4827ce-4f78-44d0-80f6-b2264cb73769 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:46.973469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4020d038-1c04-41a2-ac86-c2398f911f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:46.975601 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020d038-1c04-41a2-ac86-c2398f911f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:46.976169 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020d038-1c04-41a2-ac86-c2398f911f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:46.976660 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020d038-1c04-41a2-ac86-c2398f911f48 admin admin] PATH_INFO: `/v3/users/1494a19570144bc59bd95bcd0489230f` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:46.978609 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4020d038-1c04-41a2-ac86-c2398f911f48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1494a19570144bc59bd95bcd0489230f)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:46.985050 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4020d038-1c04-41a2-ac86-c2398f911f48 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:47.258532 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4020d038-1c04-41a2-ac86-c2398f911f48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1494a19570144bc59bd95bcd0489230f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:47.260148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4020d038-1c04-41a2-ac86-c2398f911f48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1494a19570144bc59bd95bcd0489230f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:47.264225 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4020d038-1c04-41a2-ac86-c2398f911f48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1494a19570144bc59bd95bcd0489230f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:47.291153 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4020d038-1c04-41a2-ac86-c2398f911f48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1494a19570144bc59bd95bcd0489230f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:47.301747 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4020d038-1c04-41a2-ac86-c2398f911f48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1494a19570144bc59bd95bcd0489230f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:47.358351 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508816a-653d-4a3a-a1d6-7258f94af2a6 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:47.358351 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508816a-653d-4a3a-a1d6-7258f94af2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:47.358351 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508816a-653d-4a3a-a1d6-7258f94af2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:47.386303 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1084/2175] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:48:46 2022] DELETE /identity/v3/users/1494a19570144bc59bd95bcd0489230f => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:47.394419 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d74bd-98d3-4164-8905-0e4275f3a9df None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:47.394893 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d74bd-98d3-4164-8905-0e4275f3a9df None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:47.395202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d74bd-98d3-4164-8905-0e4275f3a9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:47.404191 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d508816a-653d-4a3a-a1d6-7258f94af2a6 None None] MFA Rules not processed for user `56b3d8d145d94b358c86345eb5c3b306`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:47.405708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d508816a-653d-4a3a-a1d6-7258f94af2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:47.422069 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d68d74bd-98d3-4164-8905-0e4275f3a9df None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:47.423435 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d68d74bd-98d3-4164-8905-0e4275f3a9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:47.471578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:47.545718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1092/2176] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:48:47.586494 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1085/2177] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:47.594751 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508816a-653d-4a3a-a1d6-7258f94af2a6 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:47.599396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d508816a-653d-4a3a-a1d6-7258f94af2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:47.697462 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508816a-653d-4a3a-a1d6-7258f94af2a6 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:47.702531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d7895-c13e-4940-b47f-0b801a6d6385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:47.704664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d7895-c13e-4940-b47f-0b801a6d6385 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:47.704664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d7895-c13e-4940-b47f-0b801a6d6385 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:47.704664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d7895-c13e-4940-b47f-0b801a6d6385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:47.718407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d7895-c13e-4940-b47f-0b801a6d6385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:47.720175 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d7895-c13e-4940-b47f-0b801a6d6385 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:47.743400 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1093/2178] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:48:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:48:48.606804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d74bd-98d3-4164-8905-0e4275f3a9df None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:48.609167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d68d74bd-98d3-4164-8905-0e4275f3a9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:48.648649 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d74bd-98d3-4164-8905-0e4275f3a9df None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:48.660337 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a49e190-dad1-4845-834f-bf857a104923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:48.661589 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49e190-dad1-4845-834f-bf857a104923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:48.662026 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49e190-dad1-4845-834f-bf857a104923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:48.662570 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49e190-dad1-4845-834f-bf857a104923 admin admin] PATH_INFO: `/v3/projects/15f6694a151d4a949d48f87669af7345` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:48.665950 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a49e190-dad1-4845-834f-bf857a104923 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15f6694a151d4a949d48f87669af7345)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:48.670023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a49e190-dad1-4845-834f-bf857a104923 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:48.867937 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5a49e190-dad1-4845-834f-bf857a104923 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15f6694a151d4a949d48f87669af7345', 'request_id': 'req-5a49e190-dad1-4845-834f-bf857a104923'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:48.875923 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5a49e190-dad1-4845-834f-bf857a104923 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15f6694a151d4a949d48f87669af7345', 'request_id': 'req-5a49e190-dad1-4845-834f-bf857a104923'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:48.886355 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5a49e190-dad1-4845-834f-bf857a104923 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15f6694a151d4a949d48f87669af7345', 'request_id': 'req-5a49e190-dad1-4845-834f-bf857a104923'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:49.028133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5a49e190-dad1-4845-834f-bf857a104923 admin admin] The token cache is being invalidate because project 15f6694a151d4a949d48f87669af7345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:48:49.028418 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5a49e190-dad1-4845-834f-bf857a104923 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:49.031121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5a49e190-dad1-4845-834f-bf857a104923 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:49.033984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1086/2179] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:48:48 2022] DELETE /identity/v3/projects/15f6694a151d4a949d48f87669af7345 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:49.039171 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfaeb6-7d84-4c1f-97dd-21efe40b402b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:49.040091 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfaeb6-7d84-4c1f-97dd-21efe40b402b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:49.040521 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfaeb6-7d84-4c1f-97dd-21efe40b402b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:49.084968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a1dfaeb6-7d84-4c1f-97dd-21efe40b402b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:49.086424 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a1dfaeb6-7d84-4c1f-97dd-21efe40b402b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:49.147727 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:49.196651 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1094/2180] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:49.198045 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6674b-9aa3-4ca9-a7c1-dd576e153281 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:49.198360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6674b-9aa3-4ca9-a7c1-dd576e153281 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:49.198644 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6674b-9aa3-4ca9-a7c1-dd576e153281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:49.224583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-08b6674b-9aa3-4ca9-a7c1-dd576e153281 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:49.225861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-08b6674b-9aa3-4ca9-a7c1-dd576e153281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:49.247796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1087/2181] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:49.252427 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfaeb6-7d84-4c1f-97dd-21efe40b402b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:49.266286 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfaeb6-7d84-4c1f-97dd-21efe40b402b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:49.283383 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bbb7e-a0fa-44e9-ac0a-3ad228bb1ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:49.284114 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bbb7e-a0fa-44e9-ac0a-3ad228bb1ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:49.284401 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bbb7e-a0fa-44e9-ac0a-3ad228bb1ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:49.284685 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bbb7e-a0fa-44e9-ac0a-3ad228bb1ca1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:49.285484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455bbb7e-a0fa-44e9-ac0a-3ad228bb1ca1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:48:49.285757 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455bbb7e-a0fa-44e9-ac0a-3ad228bb1ca1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:49.287415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455bbb7e-a0fa-44e9-ac0a-3ad228bb1ca1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:49.306372 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1095/2182] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:48:49.311131 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6674b-9aa3-4ca9-a7c1-dd576e153281 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:49.321638 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6674b-9aa3-4ca9-a7c1-dd576e153281 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:49.330307 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1e752-8899-4d5e-8812-d8c44ee89de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:49.331755 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1e752-8899-4d5e-8812-d8c44ee89de1 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:49.332310 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1e752-8899-4d5e-8812-d8c44ee89de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:49.332785 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1e752-8899-4d5e-8812-d8c44ee89de1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:49.334089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1e752-8899-4d5e-8812-d8c44ee89de1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:49.338050 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1e752-8899-4d5e-8812-d8c44ee89de1 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:49.509536 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1088/2183] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:48:49 2022] POST /identity/v3/projects => generated 373 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:48:49.512429 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bbb7e-a0fa-44e9-ac0a-3ad228bb1ca1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:49.533044 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bbb7e-a0fa-44e9-ac0a-3ad228bb1ca1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:49.697902 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178b9ce-7066-42db-8aa4-29c248d2bb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:49.698664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178b9ce-7066-42db-8aa4-29c248d2bb7c admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:49.699625 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178b9ce-7066-42db-8aa4-29c248d2bb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:49.700984 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178b9ce-7066-42db-8aa4-29c248d2bb7c admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:49.701727 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178b9ce-7066-42db-8aa4-29c248d2bb7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:49.705913 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178b9ce-7066-42db-8aa4-29c248d2bb7c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:49.836062 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1096/2184] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:48:49 2022] POST /identity/v3/users => generated 287 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:48:49.857591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1e752-8899-4d5e-8812-d8c44ee89de1 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:49.902030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1e752-8899-4d5e-8812-d8c44ee89de1 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:49.908481 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b559c69-210b-4d02-bd3f-6ef27dfc7cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:49.909338 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b559c69-210b-4d02-bd3f-6ef27dfc7cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:49.909878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b559c69-210b-4d02-bd3f-6ef27dfc7cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:49.910369 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b559c69-210b-4d02-bd3f-6ef27dfc7cc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:49.911718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b559c69-210b-4d02-bd3f-6ef27dfc7cc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:49.913556 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b559c69-210b-4d02-bd3f-6ef27dfc7cc6 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:49.940899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1089/2185] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:48:49.946260 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178b9ce-7066-42db-8aa4-29c248d2bb7c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:49.972026 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178b9ce-7066-42db-8aa4-29c248d2bb7c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:49.977927 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4b09c-1bc3-4256-83c3-ff7ff8dfbe6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:49.979732 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4b09c-1bc3-4256-83c3-ff7ff8dfbe6a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:49.979732 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4b09c-1bc3-4256-83c3-ff7ff8dfbe6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:49.979732 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4b09c-1bc3-4256-83c3-ff7ff8dfbe6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:49.980808 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4b09c-1bc3-4256-83c3-ff7ff8dfbe6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:49.982360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4b09c-1bc3-4256-83c3-ff7ff8dfbe6a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:50.012006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1097/2186] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:48:50.020071 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b559c69-210b-4d02-bd3f-6ef27dfc7cc6 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:50.082266 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b559c69-210b-4d02-bd3f-6ef27dfc7cc6 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:50.084774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90474f32-1f1c-48d4-bed4-06ac60504e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:50.085727 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90474f32-1f1c-48d4-bed4-06ac60504e25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:50.086052 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90474f32-1f1c-48d4-bed4-06ac60504e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:50.086337 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90474f32-1f1c-48d4-bed4-06ac60504e25 admin admin] PATH_INFO: `/v3/projects/e03f4e1041a140edb5c56e4a67a02ce4/users/f60707026f994f97a3ba658ea9fb06a9/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:50.130570 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90474f32-1f1c-48d4-bed4-06ac60504e25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e03f4e1041a140edb5c56e4a67a02ce4, user_id=f60707026f994f97a3ba658ea9fb06a9, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:50.138348 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90474f32-1f1c-48d4-bed4-06ac60504e25 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:50.325966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1090/2187] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:48:50 2022] PUT /identity/v3/projects/e03f4e1041a140edb5c56e4a67a02ce4/users/f60707026f994f97a3ba658ea9fb06a9/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:50.344653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901e69a-b711-4995-b5a8-a1694e9acdf1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:50.345254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901e69a-b711-4995-b5a8-a1694e9acdf1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:50.345254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901e69a-b711-4995-b5a8-a1694e9acdf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:50.388732 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a901e69a-b711-4995-b5a8-a1694e9acdf1 None None] MFA Rules not processed for user `f60707026f994f97a3ba658ea9fb06a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:50.390003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a901e69a-b711-4995-b5a8-a1694e9acdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:50.488805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1098/2188] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:50.505328 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90474f32-1f1c-48d4-bed4-06ac60504e25 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:50.507313 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-90474f32-1f1c-48d4-bed4-06ac60504e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:50.572434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:50.602036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90474f32-1f1c-48d4-bed4-06ac60504e25 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:50.607893 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42f740-f2ca-4f8a-ab85-fa750f1cdc8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:50.608570 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42f740-f2ca-4f8a-ab85-fa750f1cdc8c service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:50.608852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42f740-f2ca-4f8a-ab85-fa750f1cdc8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:50.609129 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42f740-f2ca-4f8a-ab85-fa750f1cdc8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:50.618919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc42f740-f2ca-4f8a-ab85-fa750f1cdc8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:50.620496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc42f740-f2ca-4f8a-ab85-fa750f1cdc8c service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:50.633685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1091/2189] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:48:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:48:52.733847 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901e69a-b711-4995-b5a8-a1694e9acdf1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:52.735364 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a901e69a-b711-4995-b5a8-a1694e9acdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:52.781022 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:52.826071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901e69a-b711-4995-b5a8-a1694e9acdf1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:52.833019 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18536f-ae93-4981-a33b-d22d2cc52e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:52.833937 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18536f-ae93-4981-a33b-d22d2cc52e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:52.842021 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18536f-ae93-4981-a33b-d22d2cc52e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:52.842021 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18536f-ae93-4981-a33b-d22d2cc52e70 admin admin] PATH_INFO: `/v3/users/d842d6e4dedb4239a95fb8e74673bf41` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:52.842021 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18536f-ae93-4981-a33b-d22d2cc52e70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d842d6e4dedb4239a95fb8e74673bf41)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:52.842021 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18536f-ae93-4981-a33b-d22d2cc52e70 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:53.135214 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-db18536f-ae93-4981-a33b-d22d2cc52e70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd842d6e4dedb4239a95fb8e74673bf41'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:53.142708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-db18536f-ae93-4981-a33b-d22d2cc52e70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd842d6e4dedb4239a95fb8e74673bf41'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:53.144442 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-db18536f-ae93-4981-a33b-d22d2cc52e70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd842d6e4dedb4239a95fb8e74673bf41'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:53.210575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-db18536f-ae93-4981-a33b-d22d2cc52e70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd842d6e4dedb4239a95fb8e74673bf41'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:53.234923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-db18536f-ae93-4981-a33b-d22d2cc52e70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd842d6e4dedb4239a95fb8e74673bf41'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:53.332009 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1099/2190] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:48:52 2022] DELETE /identity/v3/users/d842d6e4dedb4239a95fb8e74673bf41 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:53.337016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af2b01-8e52-40dc-bfaa-63a488547caa None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:53.337607 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af2b01-8e52-40dc-bfaa-63a488547caa None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:53.337713 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af2b01-8e52-40dc-bfaa-63a488547caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:53.373303 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-70af2b01-8e52-40dc-bfaa-63a488547caa None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:53.375167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-70af2b01-8e52-40dc-bfaa-63a488547caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:53.495659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1092/2191] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:53.506764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18536f-ae93-4981-a33b-d22d2cc52e70 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:53.509089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-db18536f-ae93-4981-a33b-d22d2cc52e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:53.629760 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18536f-ae93-4981-a33b-d22d2cc52e70 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:53.638361 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c2934-6251-42cf-a21c-466a2999445c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:53.639289 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c2934-6251-42cf-a21c-466a2999445c service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:53.639838 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c2934-6251-42cf-a21c-466a2999445c service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:53.640347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c2934-6251-42cf-a21c-466a2999445c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:53.658015 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c2934-6251-42cf-a21c-466a2999445c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:53.661379 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c2934-6251-42cf-a21c-466a2999445c service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:53.694180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1100/2192] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:48:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:48:54.840087 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af2b01-8e52-40dc-bfaa-63a488547caa None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:54.850787 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-70af2b01-8e52-40dc-bfaa-63a488547caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:54.896942 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af2b01-8e52-40dc-bfaa-63a488547caa None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:54.916158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05967786-98e4-472e-b623-3642320676e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:54.922964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05967786-98e4-472e-b623-3642320676e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:54.923317 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05967786-98e4-472e-b623-3642320676e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:54.923629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05967786-98e4-472e-b623-3642320676e0 admin admin] PATH_INFO: `/v3/projects/796f946d907e4b5397667bf02fb0791c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:54.928901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05967786-98e4-472e-b623-3642320676e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=796f946d907e4b5397667bf02fb0791c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:54.938769 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05967786-98e4-472e-b623-3642320676e0 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:55.218748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-05967786-98e4-472e-b623-3642320676e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '796f946d907e4b5397667bf02fb0791c', 'request_id': 'req-05967786-98e4-472e-b623-3642320676e0'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:55.253750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-05967786-98e4-472e-b623-3642320676e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '796f946d907e4b5397667bf02fb0791c', 'request_id': 'req-05967786-98e4-472e-b623-3642320676e0'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:55.278483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-05967786-98e4-472e-b623-3642320676e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '796f946d907e4b5397667bf02fb0791c', 'request_id': 'req-05967786-98e4-472e-b623-3642320676e0'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:55.368395 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-05967786-98e4-472e-b623-3642320676e0 admin admin] The token cache is being invalidate because project 796f946d907e4b5397667bf02fb0791c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:48:55.368935 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-05967786-98e4-472e-b623-3642320676e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:55.370151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-05967786-98e4-472e-b623-3642320676e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:48:55.374582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1093/2193] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:48:54 2022] DELETE /identity/v3/projects/796f946d907e4b5397667bf02fb0791c => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:48:55.381287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2febf-07d3-4eb9-b1c6-bc70c11945d6 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:55.381776 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2febf-07d3-4eb9-b1c6-bc70c11945d6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:55.381776 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2febf-07d3-4eb9-b1c6-bc70c11945d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:55.432828 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b8f2febf-07d3-4eb9-b1c6-bc70c11945d6 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:55.434238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b8f2febf-07d3-4eb9-b1c6-bc70c11945d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:55.653947 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:55.695528 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1101/2194] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:55.704287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70f875-cc8d-469a-ab55-a607955449a0 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:55.704287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70f875-cc8d-469a-ab55-a607955449a0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:55.704287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70f875-cc8d-469a-ab55-a607955449a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:55.759263 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-ee70f875-cc8d-469a-ab55-a607955449a0 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:55.763885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ee70f875-cc8d-469a-ab55-a607955449a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:55.790371 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1094/2195] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:55.792385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2febf-07d3-4eb9-b1c6-bc70c11945d6 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:55.807552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2febf-07d3-4eb9-b1c6-bc70c11945d6 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:55.818079 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5aa89-128e-423b-bd2a-4ae3f984332a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:55.819026 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5aa89-128e-423b-bd2a-4ae3f984332a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:55.819339 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5aa89-128e-423b-bd2a-4ae3f984332a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:55.819619 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5aa89-128e-423b-bd2a-4ae3f984332a admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:55.820541 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a5aa89-128e-423b-bd2a-4ae3f984332a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:48:55.820921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a5aa89-128e-423b-bd2a-4ae3f984332a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:55.823226 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a5aa89-128e-423b-bd2a-4ae3f984332a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:55.848684 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1102/2196] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:48:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:48:55.849941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70f875-cc8d-469a-ab55-a607955449a0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:55.861454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70f875-cc8d-469a-ab55-a607955449a0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:55.873426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb138b-9d82-4ba3-be0c-b93d4474ac3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:55.876034 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb138b-9d82-4ba3-be0c-b93d4474ac3b admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:55.877268 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb138b-9d82-4ba3-be0c-b93d4474ac3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:55.877682 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb138b-9d82-4ba3-be0c-b93d4474ac3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:55.878711 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb138b-9d82-4ba3-be0c-b93d4474ac3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:55.883034 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb138b-9d82-4ba3-be0c-b93d4474ac3b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:56.453455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1095/2197] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:48:55 2022] POST /identity/v3/projects => generated 385 bytes in 604 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:48:56.470271 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5aa89-128e-423b-bd2a-4ae3f984332a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:56.473002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5aa89-128e-423b-bd2a-4ae3f984332a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:56.673154 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec704e21-d3f6-40cd-9d7b-367ca5e046f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:56.673879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec704e21-d3f6-40cd-9d7b-367ca5e046f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:56.674816 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec704e21-d3f6-40cd-9d7b-367ca5e046f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:56.675105 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec704e21-d3f6-40cd-9d7b-367ca5e046f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:56.676127 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec704e21-d3f6-40cd-9d7b-367ca5e046f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:56.678425 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec704e21-d3f6-40cd-9d7b-367ca5e046f5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:56.872056 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1103/2198] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:48:56 2022] POST /identity/v3/users => generated 293 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:48:56.874919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb138b-9d82-4ba3-be0c-b93d4474ac3b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:56.892296 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb138b-9d82-4ba3-be0c-b93d4474ac3b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:56.897667 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5190060-2926-42b1-84d1-d6110b1bbd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:56.898546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5190060-2926-42b1-84d1-d6110b1bbd86 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:56.899315 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5190060-2926-42b1-84d1-d6110b1bbd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:56.900235 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5190060-2926-42b1-84d1-d6110b1bbd86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:56.901349 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5190060-2926-42b1-84d1-d6110b1bbd86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:56.904069 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5190060-2926-42b1-84d1-d6110b1bbd86 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:56.934456 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1096/2199] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:48:56.939491 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec704e21-d3f6-40cd-9d7b-367ca5e046f5 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:56.951475 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec704e21-d3f6-40cd-9d7b-367ca5e046f5 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:56.965743 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2237d-821c-4691-a095-31b1190b0915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:56.966550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2237d-821c-4691-a095-31b1190b0915 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:56.967263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2237d-821c-4691-a095-31b1190b0915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:56.968065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2237d-821c-4691-a095-31b1190b0915 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:56.970016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2237d-821c-4691-a095-31b1190b0915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:56.972888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2237d-821c-4691-a095-31b1190b0915 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:56.987439 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1104/2200] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:48:56.991878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5190060-2926-42b1-84d1-d6110b1bbd86 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:57.005332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5190060-2926-42b1-84d1-d6110b1bbd86 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:57.012619 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef67a56-7668-4520-b87f-5ccbea54f4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:57.013825 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef67a56-7668-4520-b87f-5ccbea54f4b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:57.014265 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef67a56-7668-4520-b87f-5ccbea54f4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:57.017468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef67a56-7668-4520-b87f-5ccbea54f4b1 admin admin] PATH_INFO: `/v3/projects/0a75147b684f470890244abb41bc7d67/users/7e134a318a264a1fb870e69de07b8db8/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:57.042718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef67a56-7668-4520-b87f-5ccbea54f4b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a75147b684f470890244abb41bc7d67, user_id=7e134a318a264a1fb870e69de07b8db8, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:57.046847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef67a56-7668-4520-b87f-5ccbea54f4b1 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:57.256072 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1097/2201] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:48:56 2022] PUT /identity/v3/projects/0a75147b684f470890244abb41bc7d67/users/7e134a318a264a1fb870e69de07b8db8/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:48:57.263902 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e1812-69a3-4f70-ba72-7c3fc265c172 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:57.264432 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e1812-69a3-4f70-ba72-7c3fc265c172 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:57.264882 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e1812-69a3-4f70-ba72-7c3fc265c172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:57.293361 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ba9e1812-69a3-4f70-ba72-7c3fc265c172 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:48:57.296365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ba9e1812-69a3-4f70-ba72-7c3fc265c172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:57.440726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1105/2202] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:48:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:48:57.455085 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef67a56-7668-4520-b87f-5ccbea54f4b1 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:48:57.459187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9ef67a56-7668-4520-b87f-5ccbea54f4b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:48:57.468843 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:48:57.528626 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef67a56-7668-4520-b87f-5ccbea54f4b1 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:48:57.535305 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59490f78-da7f-4ba2-a7a8-8b43e13004c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:48:57.536259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59490f78-da7f-4ba2-a7a8-8b43e13004c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:48:57.536606 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59490f78-da7f-4ba2-a7a8-8b43e13004c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:48:57.536932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59490f78-da7f-4ba2-a7a8-8b43e13004c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:48:57.550967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59490f78-da7f-4ba2-a7a8-8b43e13004c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:48:57.552572 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59490f78-da7f-4ba2-a7a8-8b43e13004c3 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:48:57.564027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1098/2203] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:48:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:49:02.343837 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e1812-69a3-4f70-ba72-7c3fc265c172 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:02.345371 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ba9e1812-69a3-4f70-ba72-7c3fc265c172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:02.457324 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:49:02.462283 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e1812-69a3-4f70-ba72-7c3fc265c172 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:02.468256 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9228c2-0f76-4add-8bd5-6dcc6450f5b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd606fd7ef269460a8110223932f02d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:02.468939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9228c2-0f76-4add-8bd5-6dcc6450f5b3 service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:02.469231 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9228c2-0f76-4add-8bd5-6dcc6450f5b3 service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:02.469514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9228c2-0f76-4add-8bd5-6dcc6450f5b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:02.472052 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-af9228c2-0f76-4add-8bd5-6dcc6450f5b3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:02.490061 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9228c2-0f76-4add-8bd5-6dcc6450f5b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:02.493372 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9228c2-0f76-4add-8bd5-6dcc6450f5b3 service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:02.512144 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1106/2204] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:49:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:49:03.694125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59490f78-da7f-4ba2-a7a8-8b43e13004c3 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:03.695702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-59490f78-da7f-4ba2-a7a8-8b43e13004c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:03.723718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59490f78-da7f-4ba2-a7a8-8b43e13004c3 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:03.735795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd648ff-0a06-4093-bd19-79a3698d4a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:03.736959 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd648ff-0a06-4093-bd19-79a3698d4a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:03.737447 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd648ff-0a06-4093-bd19-79a3698d4a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:03.737869 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd648ff-0a06-4093-bd19-79a3698d4a43 admin admin] PATH_INFO: `/v3/users/1964f2aa98b7478ab181bcaaedc39204` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:03.740224 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd648ff-0a06-4093-bd19-79a3698d4a43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1964f2aa98b7478ab181bcaaedc39204)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:03.742759 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd648ff-0a06-4093-bd19-79a3698d4a43 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:04.055048 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6fd648ff-0a06-4093-bd19-79a3698d4a43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1964f2aa98b7478ab181bcaaedc39204'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:04.055911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6fd648ff-0a06-4093-bd19-79a3698d4a43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1964f2aa98b7478ab181bcaaedc39204'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:04.057830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6fd648ff-0a06-4093-bd19-79a3698d4a43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1964f2aa98b7478ab181bcaaedc39204'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:04.083716 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6fd648ff-0a06-4093-bd19-79a3698d4a43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1964f2aa98b7478ab181bcaaedc39204'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:04.097977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6fd648ff-0a06-4093-bd19-79a3698d4a43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1964f2aa98b7478ab181bcaaedc39204'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:04.225284 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1099/2205] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:49:03 2022] DELETE /identity/v3/users/1964f2aa98b7478ab181bcaaedc39204 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:49:04.230952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4a05d-1ed7-460c-94d6-aca3673bc1b1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:04.232162 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4a05d-1ed7-460c-94d6-aca3673bc1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:04.233128 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4a05d-1ed7-460c-94d6-aca3673bc1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:04.258367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-6dd4a05d-1ed7-460c-94d6-aca3673bc1b1 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:04.260215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6dd4a05d-1ed7-460c-94d6-aca3673bc1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:04.365242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1107/2206] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:49:04.376416 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd648ff-0a06-4093-bd19-79a3698d4a43 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:04.378425 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6fd648ff-0a06-4093-bd19-79a3698d4a43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:04.457426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd648ff-0a06-4093-bd19-79a3698d4a43 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:04.463189 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc9525-2e26-41e0-9d38-9e11f9bc41a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:04.464118 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc9525-2e26-41e0-9d38-9e11f9bc41a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:04.465164 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc9525-2e26-41e0-9d38-9e11f9bc41a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:04.465685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc9525-2e26-41e0-9d38-9e11f9bc41a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:04.477541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc9525-2e26-41e0-9d38-9e11f9bc41a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:04.479046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc9525-2e26-41e0-9d38-9e11f9bc41a2 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:04.493647 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1100/2207] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:49:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:49:05.445806 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4a05d-1ed7-460c-94d6-aca3673bc1b1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:05.446906 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6dd4a05d-1ed7-460c-94d6-aca3673bc1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:05.483602 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4a05d-1ed7-460c-94d6-aca3673bc1b1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:05.514306 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f9de7-bfb6-48c4-b856-657e9f846ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:05.516324 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f9de7-bfb6-48c4-b856-657e9f846ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:05.516847 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f9de7-bfb6-48c4-b856-657e9f846ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:05.518498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f9de7-bfb6-48c4-b856-657e9f846ed1 admin admin] PATH_INFO: `/v3/projects/1cbac3a1b7dc4042afaaebb6ad02614d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:05.521948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f9de7-bfb6-48c4-b856-657e9f846ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cbac3a1b7dc4042afaaebb6ad02614d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:05.527465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f9de7-bfb6-48c4-b856-657e9f846ed1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:05.714137 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ff1f9de7-bfb6-48c4-b856-657e9f846ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cbac3a1b7dc4042afaaebb6ad02614d', 'request_id': 'req-ff1f9de7-bfb6-48c4-b856-657e9f846ed1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:05.741846 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ff1f9de7-bfb6-48c4-b856-657e9f846ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cbac3a1b7dc4042afaaebb6ad02614d', 'request_id': 'req-ff1f9de7-bfb6-48c4-b856-657e9f846ed1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:05.757798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ff1f9de7-bfb6-48c4-b856-657e9f846ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cbac3a1b7dc4042afaaebb6ad02614d', 'request_id': 'req-ff1f9de7-bfb6-48c4-b856-657e9f846ed1'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:05.860301 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ff1f9de7-bfb6-48c4-b856-657e9f846ed1 admin admin] The token cache is being invalidate because project 1cbac3a1b7dc4042afaaebb6ad02614d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:49:05.862184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ff1f9de7-bfb6-48c4-b856-657e9f846ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:05.863167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ff1f9de7-bfb6-48c4-b856-657e9f846ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:05.865274 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1108/2208] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:49:05 2022] DELETE /identity/v3/projects/1cbac3a1b7dc4042afaaebb6ad02614d => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:49:06.002624 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc9525-2e26-41e0-9d38-9e11f9bc41a2 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:06.004254 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b9cc9525-2e26-41e0-9d38-9e11f9bc41a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:06.028354 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc9525-2e26-41e0-9d38-9e11f9bc41a2 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:06.082436 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:49:06.162909 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e54b22-245d-4ce6-ac2c-5cd1c0179c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:06.164077 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e54b22-245d-4ce6-ac2c-5cd1c0179c0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:06.164369 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e54b22-245d-4ce6-ac2c-5cd1c0179c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:06.164655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e54b22-245d-4ce6-ac2c-5cd1c0179c0a admin admin] PATH_INFO: `/v3/users/f60707026f994f97a3ba658ea9fb06a9` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:06.166254 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e54b22-245d-4ce6-ac2c-5cd1c0179c0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f60707026f994f97a3ba658ea9fb06a9)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:06.169322 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e54b22-245d-4ce6-ac2c-5cd1c0179c0a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:06.507474 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-52e54b22-245d-4ce6-ac2c-5cd1c0179c0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f60707026f994f97a3ba658ea9fb06a9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:06.508269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-52e54b22-245d-4ce6-ac2c-5cd1c0179c0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f60707026f994f97a3ba658ea9fb06a9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:06.508941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-52e54b22-245d-4ce6-ac2c-5cd1c0179c0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f60707026f994f97a3ba658ea9fb06a9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:06.528914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-52e54b22-245d-4ce6-ac2c-5cd1c0179c0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f60707026f994f97a3ba658ea9fb06a9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:06.560202 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-52e54b22-245d-4ce6-ac2c-5cd1c0179c0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f60707026f994f97a3ba658ea9fb06a9'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:06.683722 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1101/2209] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:49:06 2022] DELETE /identity/v3/users/f60707026f994f97a3ba658ea9fb06a9 => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:49:06.689084 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326f3f4-a9a7-49bf-83c2-ad9bce55f066 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:06.690742 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326f3f4-a9a7-49bf-83c2-ad9bce55f066 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:06.691939 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326f3f4-a9a7-49bf-83c2-ad9bce55f066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:06.729170 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c326f3f4-a9a7-49bf-83c2-ad9bce55f066 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:06.730696 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c326f3f4-a9a7-49bf-83c2-ad9bce55f066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:06.885571 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1109/2210] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:49:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:49:06.895452 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e54b22-245d-4ce6-ac2c-5cd1c0179c0a admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:06.901179 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-52e54b22-245d-4ce6-ac2c-5cd1c0179c0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:07.031513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e54b22-245d-4ce6-ac2c-5cd1c0179c0a admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:07.041588 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824d709-8e54-4b98-95c2-9974326d8925 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:07.045600 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824d709-8e54-4b98-95c2-9974326d8925 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:07.046051 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824d709-8e54-4b98-95c2-9974326d8925 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:07.046365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824d709-8e54-4b98-95c2-9974326d8925 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:07.064481 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4824d709-8e54-4b98-95c2-9974326d8925 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:07.066814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4824d709-8e54-4b98-95c2-9974326d8925 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:07.084059 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1102/2211] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:49:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:49:07.240627 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8bf78-bd0b-4ea4-8fba-12ced0451771 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:07.240863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8bf78-bd0b-4ea4-8fba-12ced0451771 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:07.241751 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8bf78-bd0b-4ea4-8fba-12ced0451771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:07.280394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c2c8bf78-bd0b-4ea4-8fba-12ced0451771 None None] MFA Rules not processed for user `7e134a318a264a1fb870e69de07b8db8`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:07.281766 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c2c8bf78-bd0b-4ea4-8fba-12ced0451771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:07.292551 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:49:07.361615 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a61c9-cda4-46f1-b7fe-9e4411a88983 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:07.362153 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a61c9-cda4-46f1-b7fe-9e4411a88983 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:07.362441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a61c9-cda4-46f1-b7fe-9e4411a88983 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:07.363938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1103/2212] 10.222.0.57 () {54 vars in 921 bytes} [Mon Jul 18 08:49:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:49:07.371311 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77257414-8c68-471a-ba0b-f8374e54db57 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:07.371596 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77257414-8c68-471a-ba0b-f8374e54db57 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:07.372183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77257414-8c68-471a-ba0b-f8374e54db57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:07.388713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1110/2213] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:49:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:49:07.395416 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8bf78-bd0b-4ea4-8fba-12ced0451771 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:07.398124 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c2c8bf78-bd0b-4ea4-8fba-12ced0451771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:07.402569 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-77257414-8c68-471a-ba0b-f8374e54db57 None None] MFA Rules not processed for user `84cba474fc7c470ea119de697b9d7ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:07.404290 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-77257414-8c68-471a-ba0b-f8374e54db57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:07.438534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8bf78-bd0b-4ea4-8fba-12ced0451771 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:07.460661 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70dfbd-ffa3-4ac0-b0e6-e4fe78cccb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:07.462055 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70dfbd-ffa3-4ac0-b0e6-e4fe78cccb78 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:07.462668 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70dfbd-ffa3-4ac0-b0e6-e4fe78cccb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:07.463169 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70dfbd-ffa3-4ac0-b0e6-e4fe78cccb78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:07.465185 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70dfbd-ffa3-4ac0-b0e6-e4fe78cccb78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:07.469773 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70dfbd-ffa3-4ac0-b0e6-e4fe78cccb78 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:07.520519 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1104/2214] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Jul 18 08:49:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:49:07.543997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77257414-8c68-471a-ba0b-f8374e54db57 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:07.543997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-77257414-8c68-471a-ba0b-f8374e54db57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:07.575968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1111/2215] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:49:07 2022] POST /identity/v3/projects => generated 385 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:49:07.579902 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70dfbd-ffa3-4ac0-b0e6-e4fe78cccb78 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:07.623812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77257414-8c68-471a-ba0b-f8374e54db57 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:07.646408 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:49:07.675710 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70dfbd-ffa3-4ac0-b0e6-e4fe78cccb78 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:07.682194 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18879306-efd9-4f4a-a9c7-4fef5edac5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:07.683015 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18879306-efd9-4f4a-a9c7-4fef5edac5f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:07.683446 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18879306-efd9-4f4a-a9c7-4fef5edac5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:07.684341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18879306-efd9-4f4a-a9c7-4fef5edac5f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:07.684971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18879306-efd9-4f4a-a9c7-4fef5edac5f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:07.687623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18879306-efd9-4f4a-a9c7-4fef5edac5f6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:07.694303 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8abeb9-b3e2-4204-b780-c1355c938a97 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '694088ed30b94bad9d3b16f3766252df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:07.695090 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8abeb9-b3e2-4204-b780-c1355c938a97 service glance] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:07.695367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8abeb9-b3e2-4204-b780-c1355c938a97 service glance] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:07.695637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8abeb9-b3e2-4204-b780-c1355c938a97 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:07.778813 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1112/2216] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:49:07 2022] POST /identity/v3/users => generated 299 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:49:07.779748 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18879306-efd9-4f4a-a9c7-4fef5edac5f6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:07.781723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8abeb9-b3e2-4204-b780-c1355c938a97 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:07.781937 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-18879306-efd9-4f4a-a9c7-4fef5edac5f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:07.783405 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8abeb9-b3e2-4204-b780-c1355c938a97 service glance] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:07.795237 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18879306-efd9-4f4a-a9c7-4fef5edac5f6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:07.796735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1105/2217] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:49:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:49:07.797486 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8abeb9-b3e2-4204-b780-c1355c938a97 service glance] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:07.800624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4feb9-803c-4dc1-9df4-44446d6771b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:07.801639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4feb9-803c-4dc1-9df4-44446d6771b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:07.801995 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4feb9-803c-4dc1-9df4-44446d6771b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:07.802400 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4feb9-803c-4dc1-9df4-44446d6771b4 admin admin] PATH_INFO: `/v3/projects/e03f4e1041a140edb5c56e4a67a02ce4` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:07.804169 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4feb9-803c-4dc1-9df4-44446d6771b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e03f4e1041a140edb5c56e4a67a02ce4)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:07.806197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4feb9-803c-4dc1-9df4-44446d6771b4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:07.815331 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8abeb9-b3e2-4204-b780-c1355c938a97 service glance] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:07.821907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923f296-4046-4099-8d30-dd10c1ef9873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:07.828862 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923f296-4046-4099-8d30-dd10c1ef9873 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:07.829160 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923f296-4046-4099-8d30-dd10c1ef9873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:07.829421 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923f296-4046-4099-8d30-dd10c1ef9873 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:07.830286 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923f296-4046-4099-8d30-dd10c1ef9873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:07.832716 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923f296-4046-4099-8d30-dd10c1ef9873 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:07.863918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1106/2218] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:49:07.865920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923f296-4046-4099-8d30-dd10c1ef9873 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:07.877056 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923f296-4046-4099-8d30-dd10c1ef9873 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:07.882379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328c5e7-84a7-4f66-9ba9-84f66d50bd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:07.883328 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328c5e7-84a7-4f66-9ba9-84f66d50bd0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:07.883626 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328c5e7-84a7-4f66-9ba9-84f66d50bd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:07.884222 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328c5e7-84a7-4f66-9ba9-84f66d50bd0b admin admin] PATH_INFO: `/v3/projects/f3f54d7971b84445961ce311d6e2eb21/users/60cec368ad5b4f9a93fbfdfd92f55913/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:07.908102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4328c5e7-84a7-4f66-9ba9-84f66d50bd0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3f54d7971b84445961ce311d6e2eb21, user_id=60cec368ad5b4f9a93fbfdfd92f55913, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:07.913475 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4328c5e7-84a7-4f66-9ba9-84f66d50bd0b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:08.020569 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1107/2219] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:49:07 2022] PUT /identity/v3/projects/f3f54d7971b84445961ce311d6e2eb21/users/60cec368ad5b4f9a93fbfdfd92f55913/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:49:08.024793 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328c5e7-84a7-4f66-9ba9-84f66d50bd0b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:08.038949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-03e4feb9-803c-4dc1-9df4-44446d6771b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e03f4e1041a140edb5c56e4a67a02ce4', 'request_id': 'req-03e4feb9-803c-4dc1-9df4-44446d6771b4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:08.055007 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-03e4feb9-803c-4dc1-9df4-44446d6771b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e03f4e1041a140edb5c56e4a67a02ce4', 'request_id': 'req-03e4feb9-803c-4dc1-9df4-44446d6771b4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:08.064932 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-03e4feb9-803c-4dc1-9df4-44446d6771b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e03f4e1041a140edb5c56e4a67a02ce4', 'request_id': 'req-03e4feb9-803c-4dc1-9df4-44446d6771b4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:08.164060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328c5e7-84a7-4f66-9ba9-84f66d50bd0b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:08.173977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386ff3f-53c5-4150-9428-9b15dfcd300c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:08.178861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386ff3f-53c5-4150-9428-9b15dfcd300c admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:08.178861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386ff3f-53c5-4150-9428-9b15dfcd300c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:08.178861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386ff3f-53c5-4150-9428-9b15dfcd300c admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:08.178861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386ff3f-53c5-4150-9428-9b15dfcd300c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:08.178861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386ff3f-53c5-4150-9428-9b15dfcd300c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:08.185595 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-03e4feb9-803c-4dc1-9df4-44446d6771b4 admin admin] The token cache is being invalidate because project e03f4e1041a140edb5c56e4a67a02ce4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:49:08.185904 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-03e4feb9-803c-4dc1-9df4-44446d6771b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:08.191833 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-03e4feb9-803c-4dc1-9df4-44446d6771b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:08.197640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1113/2220] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:49:07 2022] DELETE /identity/v3/projects/e03f4e1041a140edb5c56e4a67a02ce4 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:49:08.199722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c24e32-ba4b-4e80-990e-daafeda0d993 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:08.200124 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c24e32-ba4b-4e80-990e-daafeda0d993 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:08.200322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c24e32-ba4b-4e80-990e-daafeda0d993 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:08.201608 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1114/2221] 10.222.0.57 () {54 vars in 921 bytes} [Mon Jul 18 08:49:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:49:08.207846 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28419911-f4ca-4f0c-a25d-0732c978cb52 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:08.208133 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28419911-f4ca-4f0c-a25d-0732c978cb52 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:08.208595 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28419911-f4ca-4f0c-a25d-0732c978cb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:08.213959 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1108/2222] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:49:08.217106 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93854644-dff0-4223-a2cd-5924772b748b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:08.217675 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93854644-dff0-4223-a2cd-5924772b748b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:08.218107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93854644-dff0-4223-a2cd-5924772b748b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:08.240327 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-28419911-f4ca-4f0c-a25d-0732c978cb52 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:08.241897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-28419911-f4ca-4f0c-a25d-0732c978cb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:08.245219 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-93854644-dff0-4223-a2cd-5924772b748b None None] MFA Rules not processed for user `24a6b794f5774e35b67e9ec4358ffc66`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:08.247704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-93854644-dff0-4223-a2cd-5924772b748b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:08.254330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1115/2223] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:49:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:49:08.255269 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28419911-f4ca-4f0c-a25d-0732c978cb52 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:08.260864 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-28419911-f4ca-4f0c-a25d-0732c978cb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:08.280043 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28419911-f4ca-4f0c-a25d-0732c978cb52 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:08.290775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4bf42-b091-4a35-aec0-b4a579ed9872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:08.294334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4bf42-b091-4a35-aec0-b4a579ed9872 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:08.294753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4bf42-b091-4a35-aec0-b4a579ed9872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:08.295104 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4bf42-b091-4a35-aec0-b4a579ed9872 admin admin] PATH_INFO: `/v3/projects/f3f54d7971b84445961ce311d6e2eb21/users/60cec368ad5b4f9a93fbfdfd92f55913/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:08.301353 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4bf42-b091-4a35-aec0-b4a579ed9872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3f54d7971b84445961ce311d6e2eb21, user_id=60cec368ad5b4f9a93fbfdfd92f55913, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:08.304137 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4bf42-b091-4a35-aec0-b4a579ed9872 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:08.329215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1116/2224] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:49:08 2022] PUT /identity/v3/projects/f3f54d7971b84445961ce311d6e2eb21/users/60cec368ad5b4f9a93fbfdfd92f55913/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:49:08.338642 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9c51b-4088-480e-b84c-36d7864da8c8 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:08.339315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9c51b-4088-480e-b84c-36d7864da8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:08.340188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9c51b-4088-480e-b84c-36d7864da8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:08.358622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1109/2225] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Jul 18 08:49:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:49:08.371941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93854644-dff0-4223-a2cd-5924772b748b None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:08.373160 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-33d9c51b-4088-480e-b84c-36d7864da8c8 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:08.373627 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-93854644-dff0-4223-a2cd-5924772b748b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:08.375200 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-33d9c51b-4088-480e-b84c-36d7864da8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:08.387024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93854644-dff0-4223-a2cd-5924772b748b None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:08.475133 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1117/2226] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:49:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:49:08.476262 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9c51b-4088-480e-b84c-36d7864da8c8 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:08.486344 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9531666-2321-4d4f-a919-9381cbacf8ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:08.489644 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9531666-2321-4d4f-a919-9381cbacf8ed service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:08.489644 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9531666-2321-4d4f-a919-9381cbacf8ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:08.490215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9c51b-4088-480e-b84c-36d7864da8c8 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:08.490315 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9531666-2321-4d4f-a919-9381cbacf8ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:08.494776 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff03f3-d780-4d95-8b65-66fac6273e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:08.496144 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff03f3-d780-4d95-8b65-66fac6273e97 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:08.498542 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff03f3-d780-4d95-8b65-66fac6273e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:08.498542 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff03f3-d780-4d95-8b65-66fac6273e97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:08.498542 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff03f3-d780-4d95-8b65-66fac6273e97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:49:08.498542 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff03f3-d780-4d95-8b65-66fac6273e97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:08.499886 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9531666-2321-4d4f-a919-9381cbacf8ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:08.500865 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff03f3-d780-4d95-8b65-66fac6273e97 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:08.501774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9531666-2321-4d4f-a919-9381cbacf8ed service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:08.513387 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1110/2227] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:49:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:49:08.525804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1118/2228] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:49:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:49:08.534651 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9531666-2321-4d4f-a919-9381cbacf8ed service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:08.563396 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9531666-2321-4d4f-a919-9381cbacf8ed service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:08.569139 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a565f-dc42-4301-9f5e-f1cd3fe09389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:08.569850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a565f-dc42-4301-9f5e-f1cd3fe09389 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:08.570100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a565f-dc42-4301-9f5e-f1cd3fe09389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:08.570434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a565f-dc42-4301-9f5e-f1cd3fe09389 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:08.571143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a565f-dc42-4301-9f5e-f1cd3fe09389 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:08.573190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a565f-dc42-4301-9f5e-f1cd3fe09389 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:08.657747 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7021d2-1a24-4812-81c9-76c57092b3a6 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:08.659215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7021d2-1a24-4812-81c9-76c57092b3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:08.659622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7021d2-1a24-4812-81c9-76c57092b3a6 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:08.661780 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1119/2229] 10.222.0.57 () {54 vars in 921 bytes} [Mon Jul 18 08:49:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:49:08.672626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8c2b0-8c61-4de0-89c9-e5495234626a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:08.673061 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8c2b0-8c61-4de0-89c9-e5495234626a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:08.673568 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8c2b0-8c61-4de0-89c9-e5495234626a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:08.690118 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1111/2230] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:49:08 2022] POST /identity/v3/projects => generated 397 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:49:08.695746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a565f-dc42-4301-9f5e-f1cd3fe09389 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:08.719967 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-cfd8c2b0-8c61-4de0-89c9-e5495234626a None None] MFA Rules not processed for user `84cba474fc7c470ea119de697b9d7ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:08.722356 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cfd8c2b0-8c61-4de0-89c9-e5495234626a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:08.859941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1120/2231] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Jul 18 08:49:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:49:08.870657 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a565f-dc42-4301-9f5e-f1cd3fe09389 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:08.872520 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8c2b0-8c61-4de0-89c9-e5495234626a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:08.880318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c74382-f3d4-4888-9a2c-c90a59f30fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:08.881430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c74382-f3d4-4888-9a2c-c90a59f30fc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:08.881430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c74382-f3d4-4888-9a2c-c90a59f30fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:08.881729 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c74382-f3d4-4888-9a2c-c90a59f30fc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:08.882564 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c74382-f3d4-4888-9a2c-c90a59f30fc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:08.884930 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c74382-f3d4-4888-9a2c-c90a59f30fc3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:08.936282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:49:08.963579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8c2b0-8c61-4de0-89c9-e5495234626a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:08.970990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7995194-8a58-485c-863e-bd9e3c20e7e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:08.971710 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7995194-8a58-485c-863e-bd9e3c20e7e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:08.972047 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7995194-8a58-485c-863e-bd9e3c20e7e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:08.972333 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7995194-8a58-485c-863e-bd9e3c20e7e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:08.986881 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7995194-8a58-485c-863e-bd9e3c20e7e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:08.988437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7995194-8a58-485c-863e-bd9e3c20e7e4 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:09.000572 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1121/2232] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:49:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:49:09.015653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1112/2233] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:49:08 2022] POST /identity/v3/users => generated 299 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:49:09.019842 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7995194-8a58-485c-863e-bd9e3c20e7e4 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:09.031382 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7995194-8a58-485c-863e-bd9e3c20e7e4 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:09.037074 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fbda9-f8bc-46c9-840c-4a51fb80bed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:09.037247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fbda9-f8bc-46c9-840c-4a51fb80bed0 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:09.037410 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fbda9-f8bc-46c9-840c-4a51fb80bed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:09.037684 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fbda9-f8bc-46c9-840c-4a51fb80bed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:09.038434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fbda9-f8bc-46c9-840c-4a51fb80bed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:09.040109 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fbda9-f8bc-46c9-840c-4a51fb80bed0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:09.052342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1122/2234] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:49:09.061379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c74382-f3d4-4888-9a2c-c90a59f30fc3 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:09.073085 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c74382-f3d4-4888-9a2c-c90a59f30fc3 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:09.079585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8145c-8380-40cb-9772-60c4ad05e22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:09.080563 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8145c-8380-40cb-9772-60c4ad05e22e admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:09.080918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8145c-8380-40cb-9772-60c4ad05e22e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:09.081256 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8145c-8380-40cb-9772-60c4ad05e22e admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:09.082096 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8145c-8380-40cb-9772-60c4ad05e22e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:09.083940 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8145c-8380-40cb-9772-60c4ad05e22e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:09.095004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1113/2235] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:49:09.100571 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fbda9-f8bc-46c9-840c-4a51fb80bed0 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:09.112029 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fbda9-f8bc-46c9-840c-4a51fb80bed0 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:09.121053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a273e8-df02-4ff5-b476-99b77a4414c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:09.122545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a273e8-df02-4ff5-b476-99b77a4414c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:09.122545 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a273e8-df02-4ff5-b476-99b77a4414c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:09.122958 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a273e8-df02-4ff5-b476-99b77a4414c2 admin admin] PATH_INFO: `/v3/projects/b889af378edc48ba8edc4e695d7992a0/users/cf757d4109394955b9decab4c879cb0a/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:09.148628 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a273e8-df02-4ff5-b476-99b77a4414c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b889af378edc48ba8edc4e695d7992a0, user_id=cf757d4109394955b9decab4c879cb0a, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:09.168253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a273e8-df02-4ff5-b476-99b77a4414c2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:09.256205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1123/2236] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:49:09 2022] PUT /identity/v3/projects/b889af378edc48ba8edc4e695d7992a0/users/cf757d4109394955b9decab4c879cb0a/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:49:09.275849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008780b-481b-4832-93eb-ad68fa3544a6 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:09.277363 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008780b-481b-4832-93eb-ad68fa3544a6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:09.277854 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008780b-481b-4832-93eb-ad68fa3544a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:09.315462 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-8008780b-481b-4832-93eb-ad68fa3544a6 None None] MFA Rules not processed for user `cf757d4109394955b9decab4c879cb0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:09.317281 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8008780b-481b-4832-93eb-ad68fa3544a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:09.349086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:49:09.418900 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1114/2237] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:49:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:49:09.430525 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a273e8-df02-4ff5-b476-99b77a4414c2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:09.433531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-19a273e8-df02-4ff5-b476-99b77a4414c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:09.542041 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a273e8-df02-4ff5-b476-99b77a4414c2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:09.550952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c7d35-ebc3-4cbc-bc7a-b50358312f4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:09.551653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c7d35-ebc3-4cbc-bc7a-b50358312f4e service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:09.552150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c7d35-ebc3-4cbc-bc7a-b50358312f4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:09.552486 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c7d35-ebc3-4cbc-bc7a-b50358312f4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:09.564034 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c7d35-ebc3-4cbc-bc7a-b50358312f4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:09.565552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c7d35-ebc3-4cbc-bc7a-b50358312f4e service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:09.580229 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1124/2238] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:49:09 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:49:10.422508 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:49:10.750248 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:49:16.897027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18678daa-6054-4560-8bf7-356bfc9af528 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:16.898475 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18678daa-6054-4560-8bf7-356bfc9af528 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:16.898475 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18678daa-6054-4560-8bf7-356bfc9af528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:16.918726 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-18678daa-6054-4560-8bf7-356bfc9af528 None None] MFA Rules not processed for user `2b03f8b8c51e472bbd4cf0afc9c6b632`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:16.920025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-18678daa-6054-4560-8bf7-356bfc9af528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:17.013020 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1115/2239] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Jul 18 08:49:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:49:17.042115 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c7d35-ebc3-4cbc-bc7a-b50358312f4e service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:17.044685 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0f2c7d35-ebc3-4cbc-bc7a-b50358312f4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:17.136074 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c7d35-ebc3-4cbc-bc7a-b50358312f4e service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:17.143933 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a5aa3-8ab4-4367-a5a6-68f883aff757 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09a5c2e804504038817cad76544dce5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:17.145254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a5aa3-8ab4-4367-a5a6-68f883aff757 service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:17.145550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a5aa3-8ab4-4367-a5a6-68f883aff757 service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:17.145861 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a5aa3-8ab4-4367-a5a6-68f883aff757 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:17.156364 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a5aa3-8ab4-4367-a5a6-68f883aff757 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:17.158626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a5aa3-8ab4-4367-a5a6-68f883aff757 service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:17.169102 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1125/2240] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Jul 18 08:49:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 08:49:17.460570 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c34e6-3268-4eea-b9d5-ebf73406db36 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:17.460856 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c34e6-3268-4eea-b9d5-ebf73406db36 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:17.461152 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c34e6-3268-4eea-b9d5-ebf73406db36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:17.505875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-8f6c34e6-3268-4eea-b9d5-ebf73406db36 None None] MFA Rules not processed for user `60cec368ad5b4f9a93fbfdfd92f55913`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:17.508568 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8f6c34e6-3268-4eea-b9d5-ebf73406db36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:17.568210 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:49:17.627702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1116/2241] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:49:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:49:17.646463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a5aa3-8ab4-4367-a5a6-68f883aff757 service swift] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:17.649160 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-311a5aa3-8ab4-4367-a5a6-68f883aff757 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:17.743029 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a5aa3-8ab4-4367-a5a6-68f883aff757 service swift] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:17.747918 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54eace6-57b9-47a3-ad12-7eadf75e2526 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '694088ed30b94bad9d3b16f3766252df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:17.748678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54eace6-57b9-47a3-ad12-7eadf75e2526 service glance] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:17.749032 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54eace6-57b9-47a3-ad12-7eadf75e2526 service glance] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:17.749380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54eace6-57b9-47a3-ad12-7eadf75e2526 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:17.751566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b54eace6-57b9-47a3-ad12-7eadf75e2526 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:17.784289 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:49:17.866785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54eace6-57b9-47a3-ad12-7eadf75e2526 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:17.868646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54eace6-57b9-47a3-ad12-7eadf75e2526 service glance] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:17.881593 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1126/2242] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:49:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:49:18.415496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd884a-d45f-4873-914c-756f7ef25745 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:18.416231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd884a-d45f-4873-914c-756f7ef25745 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:18.416231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd884a-d45f-4873-914c-756f7ef25745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:18.444027 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-45bd884a-d45f-4873-914c-756f7ef25745 None None] MFA Rules not processed for user `2b03f8b8c51e472bbd4cf0afc9c6b632`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:18.445666 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-45bd884a-d45f-4873-914c-756f7ef25745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:18.468413 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1117/2243] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Jul 18 08:49:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:49:18.484284 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54eace6-57b9-47a3-ad12-7eadf75e2526 service glance] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:18.506325 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54eace6-57b9-47a3-ad12-7eadf75e2526 service glance] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:18.515952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56675327-b923-4368-9719-0d6b3d319f9b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09a5c2e804504038817cad76544dce5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:18.517347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56675327-b923-4368-9719-0d6b3d319f9b service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:18.517933 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56675327-b923-4368-9719-0d6b3d319f9b service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:18.518494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56675327-b923-4368-9719-0d6b3d319f9b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:18.537493 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56675327-b923-4368-9719-0d6b3d319f9b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:18.540693 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56675327-b923-4368-9719-0d6b3d319f9b service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:18.568102 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1127/2244] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Jul 18 08:49:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 08:49:19.626524 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd884a-d45f-4873-914c-756f7ef25745 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:19.748631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd884a-d45f-4873-914c-756f7ef25745 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:19.777902 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c91702-09d7-4456-9def-e38c34db4306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:19.778944 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c91702-09d7-4456-9def-e38c34db4306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:19.779234 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c91702-09d7-4456-9def-e38c34db4306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:19.779517 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c91702-09d7-4456-9def-e38c34db4306 admin admin] PATH_INFO: `/v3/users/cf757d4109394955b9decab4c879cb0a` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:19.788642 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c91702-09d7-4456-9def-e38c34db4306 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf757d4109394955b9decab4c879cb0a)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:19.796335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c91702-09d7-4456-9def-e38c34db4306 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:20.332135 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b5c91702-09d7-4456-9def-e38c34db4306 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf757d4109394955b9decab4c879cb0a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:20.335811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b5c91702-09d7-4456-9def-e38c34db4306 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf757d4109394955b9decab4c879cb0a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:20.349063 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b5c91702-09d7-4456-9def-e38c34db4306 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf757d4109394955b9decab4c879cb0a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:20.422652 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b5c91702-09d7-4456-9def-e38c34db4306 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf757d4109394955b9decab4c879cb0a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:20.471341 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b5c91702-09d7-4456-9def-e38c34db4306 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf757d4109394955b9decab4c879cb0a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:20.584829 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1118/2245] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:49:19 2022] DELETE /identity/v3/users/cf757d4109394955b9decab4c879cb0a => generated 0 bytes in 960 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:49:20.590049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3834b3a-100d-4d4b-87ea-512870a7f41b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:20.592420 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3834b3a-100d-4d4b-87ea-512870a7f41b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:20.592911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3834b3a-100d-4d4b-87ea-512870a7f41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:20.636549 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f3834b3a-100d-4d4b-87ea-512870a7f41b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:20.640731 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f3834b3a-100d-4d4b-87ea-512870a7f41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:20.791490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1128/2246] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:49:20.798821 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c91702-09d7-4456-9def-e38c34db4306 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:20.801227 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b5c91702-09d7-4456-9def-e38c34db4306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:20.912341 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:49:20.952217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c91702-09d7-4456-9def-e38c34db4306 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:20.958331 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe9099-ad90-41b5-844f-70d424a76a0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:20.960052 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe9099-ad90-41b5-844f-70d424a76a0a service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:20.962622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe9099-ad90-41b5-844f-70d424a76a0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:20.963455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe9099-ad90-41b5-844f-70d424a76a0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:20.982147 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe9099-ad90-41b5-844f-70d424a76a0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:20.984653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe9099-ad90-41b5-844f-70d424a76a0a service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:21.048853 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1119/2247] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:49:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:49:21.852564 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3834b3a-100d-4d4b-87ea-512870a7f41b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:21.854964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f3834b3a-100d-4d4b-87ea-512870a7f41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:21.883961 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3834b3a-100d-4d4b-87ea-512870a7f41b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:21.895513 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e3934-0034-4445-99fb-59b15262dc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:21.897395 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e3934-0034-4445-99fb-59b15262dc4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:21.898917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e3934-0034-4445-99fb-59b15262dc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:21.899263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e3934-0034-4445-99fb-59b15262dc4e admin admin] PATH_INFO: `/v3/projects/b889af378edc48ba8edc4e695d7992a0` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:21.901398 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e3934-0034-4445-99fb-59b15262dc4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b889af378edc48ba8edc4e695d7992a0)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:21.904200 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e3934-0034-4445-99fb-59b15262dc4e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:22.177473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ea6e3934-0034-4445-99fb-59b15262dc4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b889af378edc48ba8edc4e695d7992a0', 'request_id': 'req-ea6e3934-0034-4445-99fb-59b15262dc4e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:22.213941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ea6e3934-0034-4445-99fb-59b15262dc4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b889af378edc48ba8edc4e695d7992a0', 'request_id': 'req-ea6e3934-0034-4445-99fb-59b15262dc4e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:22.283236 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ea6e3934-0034-4445-99fb-59b15262dc4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b889af378edc48ba8edc4e695d7992a0', 'request_id': 'req-ea6e3934-0034-4445-99fb-59b15262dc4e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:22.866665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ea6e3934-0034-4445-99fb-59b15262dc4e admin admin] The token cache is being invalidate because project b889af378edc48ba8edc4e695d7992a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:49:22.867744 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ea6e3934-0034-4445-99fb-59b15262dc4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:22.868358 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ea6e3934-0034-4445-99fb-59b15262dc4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:49:22.877781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1129/2248] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:49:21 2022] DELETE /identity/v3/projects/b889af378edc48ba8edc4e695d7992a0 => generated 0 bytes in 1022 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:49:22.879051 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13c2e4-1c41-48af-970c-677319f65af9 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:22.879822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13c2e4-1c41-48af-970c-677319f65af9 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:22.879822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13c2e4-1c41-48af-970c-677319f65af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:22.913605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-ed13c2e4-1c41-48af-970c-677319f65af9 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:22.915817 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ed13c2e4-1c41-48af-970c-677319f65af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:23.074568 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1120/2249] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:49:23.076500 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c67963-6b0c-4e4c-bcea-7f6fd9f23425 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:23.076863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c67963-6b0c-4e4c-bcea-7f6fd9f23425 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:23.077231 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c67963-6b0c-4e4c-bcea-7f6fd9f23425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:23.131155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a8c67963-6b0c-4e4c-bcea-7f6fd9f23425 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:23.144379 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a8c67963-6b0c-4e4c-bcea-7f6fd9f23425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:23.174184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1130/2250] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:49:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:49:23.178661 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13c2e4-1c41-48af-970c-677319f65af9 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:23.206416 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13c2e4-1c41-48af-970c-677319f65af9 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:23.221852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e65c43-00c4-4edd-977b-8a5fd822a93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:23.222378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e65c43-00c4-4edd-977b-8a5fd822a93f admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:23.222732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e65c43-00c4-4edd-977b-8a5fd822a93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:23.223008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e65c43-00c4-4edd-977b-8a5fd822a93f admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:23.223973 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e65c43-00c4-4edd-977b-8a5fd822a93f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:49:23.224281 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e65c43-00c4-4edd-977b-8a5fd822a93f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:23.226197 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e65c43-00c4-4edd-977b-8a5fd822a93f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:23.252263 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1121/2251] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:49:23.253477 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c67963-6b0c-4e4c-bcea-7f6fd9f23425 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:23.274164 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c67963-6b0c-4e4c-bcea-7f6fd9f23425 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:23.283285 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518699a6-5820-4f37-9aab-0de7c0860482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:23.284107 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518699a6-5820-4f37-9aab-0de7c0860482 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:23.284324 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518699a6-5820-4f37-9aab-0de7c0860482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:23.284595 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518699a6-5820-4f37-9aab-0de7c0860482 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:23.285376 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518699a6-5820-4f37-9aab-0de7c0860482 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:23.287429 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518699a6-5820-4f37-9aab-0de7c0860482 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:23.315759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:49:23.525514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1131/2252] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:49:23 2022] POST /identity/v3/projects => generated 379 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:49:23.530177 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e65c43-00c4-4edd-977b-8a5fd822a93f admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:23.559833 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e65c43-00c4-4edd-977b-8a5fd822a93f admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:23.814831 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc0dda-03e1-478f-b6b7-bfaa58be8b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:23.815623 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc0dda-03e1-478f-b6b7-bfaa58be8b38 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:23.818197 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc0dda-03e1-478f-b6b7-bfaa58be8b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:23.818546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc0dda-03e1-478f-b6b7-bfaa58be8b38 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:23.819335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc0dda-03e1-478f-b6b7-bfaa58be8b38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:23.825942 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc0dda-03e1-478f-b6b7-bfaa58be8b38 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:23.982359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1122/2253] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:49:23 2022] POST /identity/v3/users => generated 290 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:49:23.986311 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518699a6-5820-4f37-9aab-0de7c0860482 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:24.027593 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518699a6-5820-4f37-9aab-0de7c0860482 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:24.060066 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed591b-d17b-406c-bc7a-82b9237d0c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:24.064786 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed591b-d17b-406c-bc7a-82b9237d0c56 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:24.069056 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed591b-d17b-406c-bc7a-82b9237d0c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:24.069878 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed591b-d17b-406c-bc7a-82b9237d0c56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:24.073956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed591b-d17b-406c-bc7a-82b9237d0c56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:24.084971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed591b-d17b-406c-bc7a-82b9237d0c56 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:24.144089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc0dda-03e1-478f-b6b7-bfaa58be8b38 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:24.155569 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1132/2254] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:49:24.166728 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc0dda-03e1-478f-b6b7-bfaa58be8b38 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:24.185015 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a379073-a657-490d-b280-034b3ff92076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:24.191263 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a379073-a657-490d-b280-034b3ff92076 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:24.191879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a379073-a657-490d-b280-034b3ff92076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:24.192426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a379073-a657-490d-b280-034b3ff92076 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:24.194334 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a379073-a657-490d-b280-034b3ff92076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:24.204793 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a379073-a657-490d-b280-034b3ff92076 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:24.267836 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1123/2255] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:49:24.271282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed591b-d17b-406c-bc7a-82b9237d0c56 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:24.288407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed591b-d17b-406c-bc7a-82b9237d0c56 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:24.295532 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743037b-96e3-4536-8b25-a8ef03551e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:24.296947 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743037b-96e3-4536-8b25-a8ef03551e1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:24.297544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743037b-96e3-4536-8b25-a8ef03551e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:24.298127 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743037b-96e3-4536-8b25-a8ef03551e1e admin admin] PATH_INFO: `/v3/projects/d37f9217ffb94786830f0f6c5970b762/users/ba4946e9565f4a608dd04fbc4de18dad/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:24.320314 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743037b-96e3-4536-8b25-a8ef03551e1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d37f9217ffb94786830f0f6c5970b762, user_id=ba4946e9565f4a608dd04fbc4de18dad, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:24.329436 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743037b-96e3-4536-8b25-a8ef03551e1e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:24.422660 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1133/2256] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:49:24 2022] PUT /identity/v3/projects/d37f9217ffb94786830f0f6c5970b762/users/ba4946e9565f4a608dd04fbc4de18dad/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:49:24.428298 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e33e10-83fe-42df-be11-cc6b318287b1 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:24.428585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e33e10-83fe-42df-be11-cc6b318287b1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:24.429561 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e33e10-83fe-42df-be11-cc6b318287b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:24.474286 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-31e33e10-83fe-42df-be11-cc6b318287b1 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:24.476986 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-31e33e10-83fe-42df-be11-cc6b318287b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:24.600949 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:49:24.643493 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1124/2257] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:49:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:49:24.655270 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743037b-96e3-4536-8b25-a8ef03551e1e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:24.658629 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f743037b-96e3-4536-8b25-a8ef03551e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:24.815657 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743037b-96e3-4536-8b25-a8ef03551e1e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:24.819587 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6555b01-de67-4e4b-a518-3db355542124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:24.820426 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6555b01-de67-4e4b-a518-3db355542124 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:24.820734 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6555b01-de67-4e4b-a518-3db355542124 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:24.821019 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6555b01-de67-4e4b-a518-3db355542124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:24.853671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6555b01-de67-4e4b-a518-3db355542124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:24.855935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6555b01-de67-4e4b-a518-3db355542124 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:24.867572 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1134/2258] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:49:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:49:34.717177 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=6 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:49:34.868951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:49:35.225727 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f72b8d-b48c-47fe-abe8-7fe252be452e None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:35.226865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f72b8d-b48c-47fe-abe8-7fe252be452e None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:35.226865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f72b8d-b48c-47fe-abe8-7fe252be452e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:35.247728 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-87f72b8d-b48c-47fe-abe8-7fe252be452e None None] MFA Rules not processed for user `7e134a318a264a1fb870e69de07b8db8`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:35.248715 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-87f72b8d-b48c-47fe-abe8-7fe252be452e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:35.324767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1125/2259] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:49:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:49:35.761282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1e30e-0983-4900-a185-69b2cb273479 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:35.761530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1e30e-0983-4900-a185-69b2cb273479 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:35.763396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1e30e-0983-4900-a185-69b2cb273479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:35.796614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-4ec1e30e-0983-4900-a185-69b2cb273479 None None] MFA Rules not processed for user `ba4946e9565f4a608dd04fbc4de18dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:35.797912 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4ec1e30e-0983-4900-a185-69b2cb273479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:35.912723 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1135/2260] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:49:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:49:35.913286 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f72b8d-b48c-47fe-abe8-7fe252be452e None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:35.923012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f72b8d-b48c-47fe-abe8-7fe252be452e None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:35.928037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990f8fd-b27b-4edb-b2cc-769e6870faa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:35.928844 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990f8fd-b27b-4edb-b2cc-769e6870faa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:35.929133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990f8fd-b27b-4edb-b2cc-769e6870faa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:35.929415 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990f8fd-b27b-4edb-b2cc-769e6870faa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:35.930105 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990f8fd-b27b-4edb-b2cc-769e6870faa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:35.932212 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990f8fd-b27b-4edb-b2cc-769e6870faa7 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:36.031136 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1126/2261] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:49:35 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:49:36.033833 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1e30e-0983-4900-a185-69b2cb273479 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:36.046565 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1e30e-0983-4900-a185-69b2cb273479 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:36.124245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3d595-0a49-4bcc-ae4a-a3095023352e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:36.125120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3d595-0a49-4bcc-ae4a-a3095023352e admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:36.125408 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3d595-0a49-4bcc-ae4a-a3095023352e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:36.125684 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3d595-0a49-4bcc-ae4a-a3095023352e admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:36.126402 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3d595-0a49-4bcc-ae4a-a3095023352e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:36.128451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3d595-0a49-4bcc-ae4a-a3095023352e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:36.240214 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1136/2262] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:49:36 2022] POST /identity/v3/users => generated 296 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:49:36.241424 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990f8fd-b27b-4edb-b2cc-769e6870faa7 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:36.255613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990f8fd-b27b-4edb-b2cc-769e6870faa7 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:36.284581 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d526d-14d6-4c2e-95f6-cab98ef4b6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:36.285790 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d526d-14d6-4c2e-95f6-cab98ef4b6b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:36.287638 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d526d-14d6-4c2e-95f6-cab98ef4b6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:36.289458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d526d-14d6-4c2e-95f6-cab98ef4b6b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:36.290807 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d526d-14d6-4c2e-95f6-cab98ef4b6b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:36.302858 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d526d-14d6-4c2e-95f6-cab98ef4b6b8 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:36.330286 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3d595-0a49-4bcc-ae4a-a3095023352e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:36.335398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1127/2263] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:49:36.349183 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3d595-0a49-4bcc-ae4a-a3095023352e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:36.359951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5234873-1727-40db-a6a2-98981171f769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:36.364145 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5234873-1727-40db-a6a2-98981171f769 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:36.364145 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5234873-1727-40db-a6a2-98981171f769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:36.364145 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5234873-1727-40db-a6a2-98981171f769 admin admin] PATH_INFO: `/v3/projects/486e45a580e64f3d8b3f063a1d19fcd7/users/bb00f4132d9247309511245686efbd43/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:36.381080 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5234873-1727-40db-a6a2-98981171f769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=486e45a580e64f3d8b3f063a1d19fcd7, user_id=bb00f4132d9247309511245686efbd43, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:36.383925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5234873-1727-40db-a6a2-98981171f769 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:36.480986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1137/2264] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:49:36 2022] PUT /identity/v3/projects/486e45a580e64f3d8b3f063a1d19fcd7/users/bb00f4132d9247309511245686efbd43/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:49:36.484947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d526d-14d6-4c2e-95f6-cab98ef4b6b8 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:36.495805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d526d-14d6-4c2e-95f6-cab98ef4b6b8 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:36.581451 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d726605-f9cf-45cc-b94d-e3d611c770a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:36.583018 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d726605-f9cf-45cc-b94d-e3d611c770a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:36.583387 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d726605-f9cf-45cc-b94d-e3d611c770a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:36.584267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d726605-f9cf-45cc-b94d-e3d611c770a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:36.585797 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d726605-f9cf-45cc-b94d-e3d611c770a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:36.587549 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d726605-f9cf-45cc-b94d-e3d611c770a6 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:36.603777 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1128/2265] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:49:36.608408 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5234873-1727-40db-a6a2-98981171f769 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:36.622763 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5234873-1727-40db-a6a2-98981171f769 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:36.631799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba16f62-26b7-474f-bbba-6d1808232354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:36.633300 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba16f62-26b7-474f-bbba-6d1808232354 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:36.633824 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba16f62-26b7-474f-bbba-6d1808232354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:36.634334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba16f62-26b7-474f-bbba-6d1808232354 admin admin] PATH_INFO: `/v3/projects/486e45a580e64f3d8b3f063a1d19fcd7/users/bb00f4132d9247309511245686efbd43/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:36.638486 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba16f62-26b7-474f-bbba-6d1808232354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=486e45a580e64f3d8b3f063a1d19fcd7, user_id=bb00f4132d9247309511245686efbd43, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:36.643499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba16f62-26b7-474f-bbba-6d1808232354 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:36.659069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1138/2266] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:49:36 2022] PUT /identity/v3/projects/486e45a580e64f3d8b3f063a1d19fcd7/users/bb00f4132d9247309511245686efbd43/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:49:42.139285 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6bc9a-20ba-4a56-b57d-4d204c42d037 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:42.140871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6bc9a-20ba-4a56-b57d-4d204c42d037 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:42.141367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6bc9a-20ba-4a56-b57d-4d204c42d037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:42.174106 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-23e6bc9a-20ba-4a56-b57d-4d204c42d037 None None] MFA Rules not processed for user `2b03f8b8c51e472bbd4cf0afc9c6b632`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:42.176774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-23e6bc9a-20ba-4a56-b57d-4d204c42d037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:42.361249 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1129/2267] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Jul 18 08:49:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:49:42.411145 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba16f62-26b7-474f-bbba-6d1808232354 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:42.413724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3ba16f62-26b7-474f-bbba-6d1808232354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:42.517455 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:49:42.571307 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba16f62-26b7-474f-bbba-6d1808232354 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:42.590253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81136221-44ab-4b0f-9ee7-f093bb324d93 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09a5c2e804504038817cad76544dce5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:42.593701 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81136221-44ab-4b0f-9ee7-f093bb324d93 service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:42.594214 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81136221-44ab-4b0f-9ee7-f093bb324d93 service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:42.594581 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81136221-44ab-4b0f-9ee7-f093bb324d93 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:42.607914 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81136221-44ab-4b0f-9ee7-f093bb324d93 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:42.610994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81136221-44ab-4b0f-9ee7-f093bb324d93 service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:42.629513 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1139/2268] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Jul 18 08:49:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 08:49:51.512228 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa060d-f598-4ec1-80dd-a954b6da0bd0 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:51.513546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa060d-f598-4ec1-80dd-a954b6da0bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:51.514004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa060d-f598-4ec1-80dd-a954b6da0bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:51.581248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:49:51.595972 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6efa060d-f598-4ec1-80dd-a954b6da0bd0 None None] MFA Rules not processed for user `bb00f4132d9247309511245686efbd43`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:51.599010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6efa060d-f598-4ec1-80dd-a954b6da0bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:51.983414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1130/2269] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:49:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:49:52.334996 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81136221-44ab-4b0f-9ee7-f093bb324d93 service swift] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:52.338304 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-81136221-44ab-4b0f-9ee7-f093bb324d93 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:52.555366 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81136221-44ab-4b0f-9ee7-f093bb324d93 service swift] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:52.562681 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd6497-5b80-45d2-8e3a-cceb76e91667 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:52.563529 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd6497-5b80-45d2-8e3a-cceb76e91667 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:52.563901 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd6497-5b80-45d2-8e3a-cceb76e91667 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:52.564460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd6497-5b80-45d2-8e3a-cceb76e91667 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:52.639448 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd6497-5b80-45d2-8e3a-cceb76e91667 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:52.641227 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd6497-5b80-45d2-8e3a-cceb76e91667 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:52.666507 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1140/2270] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:49:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:49:55.999951 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286316e3-9467-437f-9e1f-6e8488c58150 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:56.001183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286316e3-9467-437f-9e1f-6e8488c58150 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:56.001703 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286316e3-9467-437f-9e1f-6e8488c58150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:56.041155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-286316e3-9467-437f-9e1f-6e8488c58150 None None] MFA Rules not processed for user `2b03f8b8c51e472bbd4cf0afc9c6b632`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:49:56.043602 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-286316e3-9467-437f-9e1f-6e8488c58150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:49:56.065796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1131/2271] 10.222.0.57 () {58 vars in 967 bytes} [Mon Jul 18 08:49:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:49:56.099180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd6497-5b80-45d2-8e3a-cceb76e91667 service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:49:56.115092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd6497-5b80-45d2-8e3a-cceb76e91667 service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:49:56.141516 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cea5ef-4758-48e8-bc81-e5f810cb41d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09a5c2e804504038817cad76544dce5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:49:56.142466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cea5ef-4758-48e8-bc81-e5f810cb41d9 service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:49:56.142771 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cea5ef-4758-48e8-bc81-e5f810cb41d9 service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:49:56.143054 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cea5ef-4758-48e8-bc81-e5f810cb41d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:49:56.158094 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cea5ef-4758-48e8-bc81-e5f810cb41d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:49:56.160068 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cea5ef-4758-48e8-bc81-e5f810cb41d9 service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:49:56.173194 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1141/2272] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Jul 18 08:49:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 08:50:06.035275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=31 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:50:06.169071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=75 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:50:09.442843 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946080ba-cb18-4746-8459-b623106924ad None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:50:09.443730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946080ba-cb18-4746-8459-b623106924ad None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:50:09.443730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946080ba-cb18-4746-8459-b623106924ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:50:09.467211 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-946080ba-cb18-4746-8459-b623106924ad None None] MFA Rules not processed for user `4e6b788e0819483580c09902a341ea6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:50:09.468463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-946080ba-cb18-4746-8459-b623106924ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:50:09.539478 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1132/2273] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:50:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:50:16.302203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b7732-2599-4114-b236-b9f565ef432f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:50:16.304277 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b7732-2599-4114-b236-b9f565ef432f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:50:16.304277 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b7732-2599-4114-b236-b9f565ef432f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:50:16.324952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-aa4b7732-2599-4114-b236-b9f565ef432f None None] MFA Rules not processed for user `ba4946e9565f4a608dd04fbc4de18dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:50:16.326296 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-aa4b7732-2599-4114-b236-b9f565ef432f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:50:16.336604 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1142/2274] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:50:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:50:19.528081 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=25 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:50:19.961824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946080ba-cb18-4746-8459-b623106924ad None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:50:20.026734 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946080ba-cb18-4746-8459-b623106924ad None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:50:20.033127 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815056f-5975-4242-b25f-d721f6cefdbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:50:20.033928 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815056f-5975-4242-b25f-d721f6cefdbe service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:50:20.034216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815056f-5975-4242-b25f-d721f6cefdbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:50:20.034494 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815056f-5975-4242-b25f-d721f6cefdbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:50:20.036678 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c815056f-5975-4242-b25f-d721f6cefdbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:50:20.131815 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c815056f-5975-4242-b25f-d721f6cefdbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:50:20.133258 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c815056f-5975-4242-b25f-d721f6cefdbe service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:50:20.145801 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1133/2275] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:50:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:50:26.317432 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:50:30.140332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=50 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:50:35.585493 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8ccad-ab73-4add-926e-e6ed21455fa9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:50:35.585493 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8ccad-ab73-4add-926e-e6ed21455fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:50:35.585493 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8ccad-ab73-4add-926e-e6ed21455fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:50:35.603418 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-24f8ccad-ab73-4add-926e-e6ed21455fa9 None None] MFA Rules not processed for user `56b3d8d145d94b358c86345eb5c3b306`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:50:35.604700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-24f8ccad-ab73-4add-926e-e6ed21455fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:50:35.667642 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1143/2276] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:50:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:50:41.532047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815056f-5975-4242-b25f-d721f6cefdbe service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:50:41.543768 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815056f-5975-4242-b25f-d721f6cefdbe service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:50:41.550612 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443de079-231c-41c2-95e2-5c0a3e975ca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:50:41.551549 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443de079-231c-41c2-95e2-5c0a3e975ca7 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:50:41.551942 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443de079-231c-41c2-95e2-5c0a3e975ca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:50:41.552231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443de079-231c-41c2-95e2-5c0a3e975ca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:50:41.566245 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443de079-231c-41c2-95e2-5c0a3e975ca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:50:41.568424 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443de079-231c-41c2-95e2-5c0a3e975ca7 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:50:41.580568 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1134/2277] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:50:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:50:45.660444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=25 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:50:51.573842 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=6 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:50:53.224015 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbff8e5-2c81-4e2b-b419-0dfe3da6e3a1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:50:53.224531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbff8e5-2c81-4e2b-b419-0dfe3da6e3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:50:53.224626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbff8e5-2c81-4e2b-b419-0dfe3da6e3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:50:53.244300 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7cbff8e5-2c81-4e2b-b419-0dfe3da6e3a1 None None] MFA Rules not processed for user `88b4bd52ede644f0aac8dff94e09dd7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:50:53.245591 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7cbff8e5-2c81-4e2b-b419-0dfe3da6e3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:50:53.303953 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1144/2278] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:50:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:50:59.869080 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60868548-0145-4ab2-997a-c5bfaa3b8f6d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:50:59.869666 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60868548-0145-4ab2-997a-c5bfaa3b8f6d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:50:59.869666 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60868548-0145-4ab2-997a-c5bfaa3b8f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:50:59.893763 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-60868548-0145-4ab2-997a-c5bfaa3b8f6d None None] MFA Rules not processed for user `94eed85a730b46dcb1cf694a3ebf2d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:50:59.895090 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-60868548-0145-4ab2-997a-c5bfaa3b8f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:50:59.981136 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1135/2279] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:50:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:51:03.296573 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=25 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:51:09.972340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=25 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:51:20.871271 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbff8e5-2c81-4e2b-b419-0dfe3da6e3a1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:20.872713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7cbff8e5-2c81-4e2b-b419-0dfe3da6e3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:20.928658 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbff8e5-2c81-4e2b-b419-0dfe3da6e3a1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:20.932221 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2c4d4-0661-4066-9c3c-6a0d1fd88ab4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:20.932870 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2c4d4-0661-4066-9c3c-6a0d1fd88ab4 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:20.933161 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2c4d4-0661-4066-9c3c-6a0d1fd88ab4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:20.933435 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2c4d4-0661-4066-9c3c-6a0d1fd88ab4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:20.935206 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7de2c4d4-0661-4066-9c3c-6a0d1fd88ab4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:20.947513 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de2c4d4-0661-4066-9c3c-6a0d1fd88ab4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:20.948995 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de2c4d4-0661-4066-9c3c-6a0d1fd88ab4 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:20.959038 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1145/2280] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:51:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:51:23.430938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60868548-0145-4ab2-997a-c5bfaa3b8f6d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:23.432439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-60868548-0145-4ab2-997a-c5bfaa3b8f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:23.445112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60868548-0145-4ab2-997a-c5bfaa3b8f6d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:23.448754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f8965-b9ad-49c5-9cd7-4caecccf419c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:23.449405 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f8965-b9ad-49c5-9cd7-4caecccf419c service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:23.449690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f8965-b9ad-49c5-9cd7-4caecccf419c service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:23.449959 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f8965-b9ad-49c5-9cd7-4caecccf419c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:23.451761 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-929f8965-b9ad-49c5-9cd7-4caecccf419c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:23.465591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f8965-b9ad-49c5-9cd7-4caecccf419c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:23.467042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f8965-b9ad-49c5-9cd7-4caecccf419c service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:23.478020 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1136/2281] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:51:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:51:24.284519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2c4d4-0661-4066-9c3c-6a0d1fd88ab4 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:24.293275 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2c4d4-0661-4066-9c3c-6a0d1fd88ab4 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:24.296708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96d9d5-7a0e-4256-a045-1d61a8f80365 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:24.297341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96d9d5-7a0e-4256-a045-1d61a8f80365 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:24.297622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96d9d5-7a0e-4256-a045-1d61a8f80365 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:24.298420 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96d9d5-7a0e-4256-a045-1d61a8f80365 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:24.300367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-5f96d9d5-7a0e-4256-a045-1d61a8f80365 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:24.314722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f96d9d5-7a0e-4256-a045-1d61a8f80365 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:24.316205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f96d9d5-7a0e-4256-a045-1d61a8f80365 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:24.327752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1146/2282] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:51:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:51:24.498668 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f8965-b9ad-49c5-9cd7-4caecccf419c service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:24.509963 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f8965-b9ad-49c5-9cd7-4caecccf419c service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:24.514765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163b83c-1ebd-4434-a1ec-4a1267bc15e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:24.515061 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163b83c-1ebd-4434-a1ec-4a1267bc15e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:24.515321 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163b83c-1ebd-4434-a1ec-4a1267bc15e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:24.515798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163b83c-1ebd-4434-a1ec-4a1267bc15e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:24.518193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5163b83c-1ebd-4434-a1ec-4a1267bc15e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:24.534462 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163b83c-1ebd-4434-a1ec-4a1267bc15e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:24.535984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163b83c-1ebd-4434-a1ec-4a1267bc15e5 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:24.549010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1137/2283] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:51:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:51:34.322688 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=32 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:51:34.542766 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=12 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:51:38.892133 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef14709-6944-4bba-bee8-b12a00c2d275 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:38.893139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef14709-6944-4bba-bee8-b12a00c2d275 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:38.893296 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef14709-6944-4bba-bee8-b12a00c2d275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:38.911875 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7ef14709-6944-4bba-bee8-b12a00c2d275 None None] MFA Rules not processed for user `4e6b788e0819483580c09902a341ea6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:51:38.913084 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7ef14709-6944-4bba-bee8-b12a00c2d275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:38.920133 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1147/2284] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:51:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:51:39.232132 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163b83c-1ebd-4434-a1ec-4a1267bc15e5 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:39.241734 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163b83c-1ebd-4434-a1ec-4a1267bc15e5 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:39.245358 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004d619-9070-423e-bfd9-9f61c599da96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:39.246000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004d619-9070-423e-bfd9-9f61c599da96 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:39.246281 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004d619-9070-423e-bfd9-9f61c599da96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:39.246554 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004d619-9070-423e-bfd9-9f61c599da96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:39.248373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5004d619-9070-423e-bfd9-9f61c599da96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:39.258648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5004d619-9070-423e-bfd9-9f61c599da96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:39.260112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5004d619-9070-423e-bfd9-9f61c599da96 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:39.270432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1138/2285] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:51:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:51:39.693722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef14709-6944-4bba-bee8-b12a00c2d275 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:39.693722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7ef14709-6944-4bba-bee8-b12a00c2d275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:39.771604 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef14709-6944-4bba-bee8-b12a00c2d275 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:39.776624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82429776-db77-4e94-8626-b9d44e72466e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd606fd7ef269460a8110223932f02d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:39.777275 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82429776-db77-4e94-8626-b9d44e72466e service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:39.777561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82429776-db77-4e94-8626-b9d44e72466e service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:39.777837 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82429776-db77-4e94-8626-b9d44e72466e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:39.779653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-82429776-db77-4e94-8626-b9d44e72466e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:39.792428 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82429776-db77-4e94-8626-b9d44e72466e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:39.794084 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82429776-db77-4e94-8626-b9d44e72466e service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:39.804131 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1148/2286] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:51:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:51:43.501648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004d619-9070-423e-bfd9-9f61c599da96 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:43.512741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004d619-9070-423e-bfd9-9f61c599da96 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:43.519743 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ad452-f3a7-4226-8e1e-17ac8ad663d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:43.520918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ad452-f3a7-4226-8e1e-17ac8ad663d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:43.521640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ad452-f3a7-4226-8e1e-17ac8ad663d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:43.522307 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ad452-f3a7-4226-8e1e-17ac8ad663d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:43.528107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bb2ad452-f3a7-4226-8e1e-17ac8ad663d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:43.538931 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ad452-f3a7-4226-8e1e-17ac8ad663d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:43.540543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ad452-f3a7-4226-8e1e-17ac8ad663d7 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:43.551000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1139/2287] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:51:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:51:44.647416 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82429776-db77-4e94-8626-b9d44e72466e service placement] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:44.662293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82429776-db77-4e94-8626-b9d44e72466e service placement] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:44.675869 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e84b46-48f4-43ff-99ee-76b4e1a98b3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:44.676799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e84b46-48f4-43ff-99ee-76b4e1a98b3e service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:44.677247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e84b46-48f4-43ff-99ee-76b4e1a98b3e service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:44.677518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e84b46-48f4-43ff-99ee-76b4e1a98b3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:44.679421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-80e84b46-48f4-43ff-99ee-76b4e1a98b3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:44.693111 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e84b46-48f4-43ff-99ee-76b4e1a98b3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:44.694776 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e84b46-48f4-43ff-99ee-76b4e1a98b3e service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:44.707258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1149/2288] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:51:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:51:50.938864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ad452-f3a7-4226-8e1e-17ac8ad663d7 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:50.949190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ad452-f3a7-4226-8e1e-17ac8ad663d7 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:50.953130 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87edb071-1bd6-4545-8f43-38ec3c783fbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:50.953760 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edb071-1bd6-4545-8f43-38ec3c783fbb service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:50.954168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edb071-1bd6-4545-8f43-38ec3c783fbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:50.954437 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edb071-1bd6-4545-8f43-38ec3c783fbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:50.956870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-87edb071-1bd6-4545-8f43-38ec3c783fbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:51.028684 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87edb071-1bd6-4545-8f43-38ec3c783fbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:51.030171 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87edb071-1bd6-4545-8f43-38ec3c783fbb service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:51.041685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1140/2289] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:51:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:51:54.201674 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e84b46-48f4-43ff-99ee-76b4e1a98b3e service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:54.214147 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e84b46-48f4-43ff-99ee-76b4e1a98b3e service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:54.220222 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136b77a-5c7f-4c54-82a0-44cd02f8fbaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:54.220948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136b77a-5c7f-4c54-82a0-44cd02f8fbaa service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:54.221212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136b77a-5c7f-4c54-82a0-44cd02f8fbaa service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:54.221557 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136b77a-5c7f-4c54-82a0-44cd02f8fbaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:54.223391 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d136b77a-5c7f-4c54-82a0-44cd02f8fbaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:54.234601 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136b77a-5c7f-4c54-82a0-44cd02f8fbaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:54.238814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136b77a-5c7f-4c54-82a0-44cd02f8fbaa service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:54.246341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1150/2290] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:51:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:51:54.943972 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87edb071-1bd6-4545-8f43-38ec3c783fbb service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:54.956721 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87edb071-1bd6-4545-8f43-38ec3c783fbb service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:54.961915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5194dc83-6a51-4349-9b36-f0261442059e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd606fd7ef269460a8110223932f02d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:54.962561 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194dc83-6a51-4349-9b36-f0261442059e service placement] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:54.962844 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194dc83-6a51-4349-9b36-f0261442059e service placement] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:54.963118 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194dc83-6a51-4349-9b36-f0261442059e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:54.964949 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5194dc83-6a51-4349-9b36-f0261442059e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:54.981080 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5194dc83-6a51-4349-9b36-f0261442059e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:54.982703 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5194dc83-6a51-4349-9b36-f0261442059e service placement] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:54.994541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1141/2291] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:51:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:51:55.120924 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136b77a-5c7f-4c54-82a0-44cd02f8fbaa service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:55.122435 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d136b77a-5c7f-4c54-82a0-44cd02f8fbaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:55.138634 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136b77a-5c7f-4c54-82a0-44cd02f8fbaa service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:55.144269 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c7f72-cb52-4155-a0a5-cd332f685eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:55.145593 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c7f72-cb52-4155-a0a5-cd332f685eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:55.145879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c7f72-cb52-4155-a0a5-cd332f685eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:55.146325 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c7f72-cb52-4155-a0a5-cd332f685eac admin admin] PATH_INFO: `/v3/users/56b3d8d145d94b358c86345eb5c3b306` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:55.147914 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c7f72-cb52-4155-a0a5-cd332f685eac admin admin] RBAC: Authorizing `identity:delete_user(user_id=56b3d8d145d94b358c86345eb5c3b306)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:55.150047 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c7f72-cb52-4155-a0a5-cd332f685eac admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:55.390724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-375c7f72-cb52-4155-a0a5-cd332f685eac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56b3d8d145d94b358c86345eb5c3b306'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:51:55.391482 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-375c7f72-cb52-4155-a0a5-cd332f685eac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56b3d8d145d94b358c86345eb5c3b306'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:51:55.392195 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-375c7f72-cb52-4155-a0a5-cd332f685eac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56b3d8d145d94b358c86345eb5c3b306'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:51:55.406909 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-375c7f72-cb52-4155-a0a5-cd332f685eac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56b3d8d145d94b358c86345eb5c3b306'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:51:55.414111 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-375c7f72-cb52-4155-a0a5-cd332f685eac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56b3d8d145d94b358c86345eb5c3b306'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:51:55.511376 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1151/2292] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:51:55 2022] DELETE /identity/v3/users/56b3d8d145d94b358c86345eb5c3b306 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:51:55.837155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5194dc83-6a51-4349-9b36-f0261442059e service placement] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:55.851354 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5194dc83-6a51-4349-9b36-f0261442059e service placement] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:55.921922 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f3be5-934a-49aa-ac16-2f7e1f6e0476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:55.922790 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f3be5-934a-49aa-ac16-2f7e1f6e0476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:55.923080 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f3be5-934a-49aa-ac16-2f7e1f6e0476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:55.923359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f3be5-934a-49aa-ac16-2f7e1f6e0476 admin admin] PATH_INFO: `/v3/projects/c2f44b51cc624637bccd9ed1d3b1a769` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:55.925217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f3be5-934a-49aa-ac16-2f7e1f6e0476 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2f44b51cc624637bccd9ed1d3b1a769)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:55.927184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f3be5-934a-49aa-ac16-2f7e1f6e0476 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:56.052293 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-215f3be5-934a-49aa-ac16-2f7e1f6e0476 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2f44b51cc624637bccd9ed1d3b1a769', 'request_id': 'req-215f3be5-934a-49aa-ac16-2f7e1f6e0476'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:51:56.058001 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:51:56.059384 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-215f3be5-934a-49aa-ac16-2f7e1f6e0476 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2f44b51cc624637bccd9ed1d3b1a769', 'request_id': 'req-215f3be5-934a-49aa-ac16-2f7e1f6e0476'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:51:56.068001 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-215f3be5-934a-49aa-ac16-2f7e1f6e0476 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2f44b51cc624637bccd9ed1d3b1a769', 'request_id': 'req-215f3be5-934a-49aa-ac16-2f7e1f6e0476'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:51:56.163213 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-215f3be5-934a-49aa-ac16-2f7e1f6e0476 admin admin] The token cache is being invalidate because project c2f44b51cc624637bccd9ed1d3b1a769 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:51:56.163507 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-215f3be5-934a-49aa-ac16-2f7e1f6e0476 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:51:56.164199 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-215f3be5-934a-49aa-ac16-2f7e1f6e0476 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:51:56.165936 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1142/2293] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:51:55 2022] DELETE /identity/v3/projects/c2f44b51cc624637bccd9ed1d3b1a769 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:51:56.172108 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f1966-8a7c-40f5-9853-bc31210a0b94 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:56.172399 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f1966-8a7c-40f5-9853-bc31210a0b94 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:56.172680 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f1966-8a7c-40f5-9853-bc31210a0b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:56.193150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-471f1966-8a7c-40f5-9853-bc31210a0b94 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:51:56.194500 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-471f1966-8a7c-40f5-9853-bc31210a0b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:56.222393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:51:56.259481 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1152/2294] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:51:56.271396 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7db1b2-437a-478e-a294-8e81f0923b51 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:56.271867 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7db1b2-437a-478e-a294-8e81f0923b51 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:56.272009 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7db1b2-437a-478e-a294-8e81f0923b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:56.292581 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-8f7db1b2-437a-478e-a294-8e81f0923b51 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:51:56.293876 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8f7db1b2-437a-478e-a294-8e81f0923b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:56.302933 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1143/2295] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:51:56.307342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f1966-8a7c-40f5-9853-bc31210a0b94 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:56.322473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f1966-8a7c-40f5-9853-bc31210a0b94 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:56.324710 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f957b85-32da-4410-8a5b-46905a65f669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:56.325378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f957b85-32da-4410-8a5b-46905a65f669 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:56.325660 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f957b85-32da-4410-8a5b-46905a65f669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:56.325937 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f957b85-32da-4410-8a5b-46905a65f669 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:56.326910 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f957b85-32da-4410-8a5b-46905a65f669 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:51:56.327333 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f957b85-32da-4410-8a5b-46905a65f669 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:56.329071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f957b85-32da-4410-8a5b-46905a65f669 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:56.345171 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1153/2296] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:51:56.347114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7db1b2-437a-478e-a294-8e81f0923b51 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:56.357320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7db1b2-437a-478e-a294-8e81f0923b51 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:56.363086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c6112-247b-4e38-b362-631dae1befd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:56.363829 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c6112-247b-4e38-b362-631dae1befd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:56.364114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c6112-247b-4e38-b362-631dae1befd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:56.364389 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c6112-247b-4e38-b362-631dae1befd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:56.365233 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c6112-247b-4e38-b362-631dae1befd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:56.366992 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c6112-247b-4e38-b362-631dae1befd3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:56.445103 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f957b85-32da-4410-8a5b-46905a65f669 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:56.447084 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9f957b85-32da-4410-8a5b-46905a65f669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:56.516923 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1144/2297] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:51:56 2022] POST /identity/v3/projects => generated 367 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:51:56.518468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c6112-247b-4e38-b362-631dae1befd3 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:56.529393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f957b85-32da-4410-8a5b-46905a65f669 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:56.613713 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c6112-247b-4e38-b362-631dae1befd3 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:56.618672 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edff5a4-cd4c-40d2-bec0-5c4d0f8a0c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:56.619556 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edff5a4-cd4c-40d2-bec0-5c4d0f8a0c5f admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:56.619892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edff5a4-cd4c-40d2-bec0-5c4d0f8a0c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:56.620174 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edff5a4-cd4c-40d2-bec0-5c4d0f8a0c5f admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:56.620935 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edff5a4-cd4c-40d2-bec0-5c4d0f8a0c5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:56.622616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a1abe-5417-4532-9c72-43a01b1126ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:56.623172 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edff5a4-cd4c-40d2-bec0-5c4d0f8a0c5f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:56.625223 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a1abe-5417-4532-9c72-43a01b1126ca service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:56.625518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a1abe-5417-4532-9c72-43a01b1126ca service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:56.625806 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a1abe-5417-4532-9c72-43a01b1126ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:56.628604 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-779a1abe-5417-4532-9c72-43a01b1126ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:56.643982 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a1abe-5417-4532-9c72-43a01b1126ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:56.646039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a1abe-5417-4532-9c72-43a01b1126ca service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:56.660058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1154/2298] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:51:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:51:56.741315 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1145/2299] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:51:56 2022] POST /identity/v3/users => generated 284 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:51:56.745812 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a1abe-5417-4532-9c72-43a01b1126ca service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:56.757785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a1abe-5417-4532-9c72-43a01b1126ca service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:56.763174 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e4025-19f8-43cd-b35c-048487590aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:56.763892 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e4025-19f8-43cd-b35c-048487590aaf admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:56.764185 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e4025-19f8-43cd-b35c-048487590aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:56.764466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e4025-19f8-43cd-b35c-048487590aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:56.765215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e4025-19f8-43cd-b35c-048487590aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:56.767102 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e4025-19f8-43cd-b35c-048487590aaf admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:56.781416 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1155/2300] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:51:56.786468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edff5a4-cd4c-40d2-bec0-5c4d0f8a0c5f admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:56.800140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edff5a4-cd4c-40d2-bec0-5c4d0f8a0c5f admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:56.807065 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa8f39-bf4a-437d-a930-cbe70c4fbb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:56.808028 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa8f39-bf4a-437d-a930-cbe70c4fbb60 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:56.808386 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa8f39-bf4a-437d-a930-cbe70c4fbb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:56.808748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa8f39-bf4a-437d-a930-cbe70c4fbb60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:56.809564 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa8f39-bf4a-437d-a930-cbe70c4fbb60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:56.811477 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa8f39-bf4a-437d-a930-cbe70c4fbb60 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:56.824802 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1146/2301] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:51:56.829187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e4025-19f8-43cd-b35c-048487590aaf admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:56.847278 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e4025-19f8-43cd-b35c-048487590aaf admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:56.847977 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4534abfc-e67d-4f99-9668-b673c9130488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:56.848972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534abfc-e67d-4f99-9668-b673c9130488 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:56.849336 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534abfc-e67d-4f99-9668-b673c9130488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:56.849725 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534abfc-e67d-4f99-9668-b673c9130488 admin admin] PATH_INFO: `/v3/projects/b1bb43614107453b9eaeb1ded517795f/users/1903dfbff98b46c79f9ac33aea9fa3d7/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:56.872215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4534abfc-e67d-4f99-9668-b673c9130488 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1bb43614107453b9eaeb1ded517795f, user_id=1903dfbff98b46c79f9ac33aea9fa3d7, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:56.875223 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4534abfc-e67d-4f99-9668-b673c9130488 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:56.973169 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1156/2302] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:51:56 2022] PUT /identity/v3/projects/b1bb43614107453b9eaeb1ded517795f/users/1903dfbff98b46c79f9ac33aea9fa3d7/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:51:56.984046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd983367-e7aa-4907-9be6-f4ed91588656 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:56.984435 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd983367-e7aa-4907-9be6-f4ed91588656 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:56.984631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd983367-e7aa-4907-9be6-f4ed91588656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:57.029203 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-fd983367-e7aa-4907-9be6-f4ed91588656 None None] MFA Rules not processed for user `1903dfbff98b46c79f9ac33aea9fa3d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:51:57.030454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-fd983367-e7aa-4907-9be6-f4ed91588656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:57.120729 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1147/2303] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:51:56 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:51:57.133236 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4534abfc-e67d-4f99-9668-b673c9130488 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:57.134999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4534abfc-e67d-4f99-9668-b673c9130488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:57.145199 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:51:57.210607 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4534abfc-e67d-4f99-9668-b673c9130488 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:57.215075 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406df46-7eb1-4790-a0bf-d4b2d4f69a76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:57.215996 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406df46-7eb1-4790-a0bf-d4b2d4f69a76 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:57.216283 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406df46-7eb1-4790-a0bf-d4b2d4f69a76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:57.219410 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406df46-7eb1-4790-a0bf-d4b2d4f69a76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:57.228532 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406df46-7eb1-4790-a0bf-d4b2d4f69a76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:57.229998 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406df46-7eb1-4790-a0bf-d4b2d4f69a76 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:57.247684 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1157/2304] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:51:57 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:51:58.769423 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd983367-e7aa-4907-9be6-f4ed91588656 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:51:58.797539 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:51:58.843731 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd983367-e7aa-4907-9be6-f4ed91588656 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:51:58.849762 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a254a-cec0-4e82-ab6d-f31ff6965140 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:51:58.850425 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a254a-cec0-4e82-ab6d-f31ff6965140 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:51:58.850713 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a254a-cec0-4e82-ab6d-f31ff6965140 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:51:58.850987 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a254a-cec0-4e82-ab6d-f31ff6965140 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:51:58.853495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e95a254a-cec0-4e82-ab6d-f31ff6965140 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:51:58.917449 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a254a-cec0-4e82-ab6d-f31ff6965140 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:51:58.919089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a254a-cec0-4e82-ab6d-f31ff6965140 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:51:58.929045 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1148/2305] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:51:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:52:07.318178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=24 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:07.507622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406df46-7eb1-4790-a0bf-d4b2d4f69a76 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:07.575929 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406df46-7eb1-4790-a0bf-d4b2d4f69a76 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:07.581811 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3fea18-34e2-4b75-bce9-58fc5fd208cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:07.582709 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3fea18-34e2-4b75-bce9-58fc5fd208cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:07.582986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3fea18-34e2-4b75-bce9-58fc5fd208cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:07.583270 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3fea18-34e2-4b75-bce9-58fc5fd208cf admin admin] PATH_INFO: `/v3/users/1903dfbff98b46c79f9ac33aea9fa3d7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:07.584535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3fea18-34e2-4b75-bce9-58fc5fd208cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1903dfbff98b46c79f9ac33aea9fa3d7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:07.586726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3fea18-34e2-4b75-bce9-58fc5fd208cf admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:07.853859 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6e3fea18-34e2-4b75-bce9-58fc5fd208cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1903dfbff98b46c79f9ac33aea9fa3d7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:07.854544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6e3fea18-34e2-4b75-bce9-58fc5fd208cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1903dfbff98b46c79f9ac33aea9fa3d7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:07.855160 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6e3fea18-34e2-4b75-bce9-58fc5fd208cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1903dfbff98b46c79f9ac33aea9fa3d7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:07.869134 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6e3fea18-34e2-4b75-bce9-58fc5fd208cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1903dfbff98b46c79f9ac33aea9fa3d7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:07.876024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6e3fea18-34e2-4b75-bce9-58fc5fd208cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1903dfbff98b46c79f9ac33aea9fa3d7'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:07.950595 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1158/2306] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:52:07 2022] DELETE /identity/v3/users/1903dfbff98b46c79f9ac33aea9fa3d7 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:07.956688 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e0b2c-4a9d-4af7-8edb-274c845246d6 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:07.957265 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e0b2c-4a9d-4af7-8edb-274c845246d6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:07.957681 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e0b2c-4a9d-4af7-8edb-274c845246d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:07.978908 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-ca3e0b2c-4a9d-4af7-8edb-274c845246d6 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:07.980271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ca3e0b2c-4a9d-4af7-8edb-274c845246d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:08.053754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1149/2307] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:08.064494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3fea18-34e2-4b75-bce9-58fc5fd208cf admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:08.066354 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6e3fea18-34e2-4b75-bce9-58fc5fd208cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:08.136904 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3fea18-34e2-4b75-bce9-58fc5fd208cf admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:08.141435 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b1303-52f8-40dc-b5df-6439899240e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:08.143973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b1303-52f8-40dc-b5df-6439899240e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:08.143973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b1303-52f8-40dc-b5df-6439899240e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:08.143973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b1303-52f8-40dc-b5df-6439899240e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:08.181237 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b1303-52f8-40dc-b5df-6439899240e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:08.184197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b1303-52f8-40dc-b5df-6439899240e6 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:08.195897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1159/2308] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:52:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:52:08.709858 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e0b2c-4a9d-4af7-8edb-274c845246d6 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:08.711376 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ca3e0b2c-4a9d-4af7-8edb-274c845246d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:08.737687 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e0b2c-4a9d-4af7-8edb-274c845246d6 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:08.743123 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a2561-e94e-459f-a2ef-99d17ff167b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:08.744141 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a2561-e94e-459f-a2ef-99d17ff167b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:08.744407 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a2561-e94e-459f-a2ef-99d17ff167b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:08.744807 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a2561-e94e-459f-a2ef-99d17ff167b2 admin admin] PATH_INFO: `/v3/projects/b1bb43614107453b9eaeb1ded517795f` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:08.746256 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a2561-e94e-459f-a2ef-99d17ff167b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1bb43614107453b9eaeb1ded517795f)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:08.748636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a2561-e94e-459f-a2ef-99d17ff167b2 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:08.906901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9c8a2561-e94e-459f-a2ef-99d17ff167b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1bb43614107453b9eaeb1ded517795f', 'request_id': 'req-9c8a2561-e94e-459f-a2ef-99d17ff167b2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:08.912545 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:08.913782 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9c8a2561-e94e-459f-a2ef-99d17ff167b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1bb43614107453b9eaeb1ded517795f', 'request_id': 'req-9c8a2561-e94e-459f-a2ef-99d17ff167b2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:08.920929 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9c8a2561-e94e-459f-a2ef-99d17ff167b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1bb43614107453b9eaeb1ded517795f', 'request_id': 'req-9c8a2561-e94e-459f-a2ef-99d17ff167b2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:09.055426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9c8a2561-e94e-459f-a2ef-99d17ff167b2 admin admin] The token cache is being invalidate because project b1bb43614107453b9eaeb1ded517795f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:52:09.056204 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9c8a2561-e94e-459f-a2ef-99d17ff167b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:09.060332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9c8a2561-e94e-459f-a2ef-99d17ff167b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:09.062993 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1150/2309] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:52:08 2022] DELETE /identity/v3/projects/b1bb43614107453b9eaeb1ded517795f => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:52:09.071617 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a602253-1121-4691-b7ed-84a9967107d4 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:09.072180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a602253-1121-4691-b7ed-84a9967107d4 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:09.072300 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a602253-1121-4691-b7ed-84a9967107d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:09.093349 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7a602253-1121-4691-b7ed-84a9967107d4 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:09.094729 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7a602253-1121-4691-b7ed-84a9967107d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:09.131711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:09.184032 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1160/2310] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:09.194112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0adaf3-a3c1-4735-82c9-8a5300535822 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:09.194670 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0adaf3-a3c1-4735-82c9-8a5300535822 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:09.195034 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0adaf3-a3c1-4735-82c9-8a5300535822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:09.217332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-ee0adaf3-a3c1-4735-82c9-8a5300535822 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:09.218575 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ee0adaf3-a3c1-4735-82c9-8a5300535822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:09.227566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1151/2311] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:09.233504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a602253-1121-4691-b7ed-84a9967107d4 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:09.245909 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a602253-1121-4691-b7ed-84a9967107d4 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:09.252626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cfeea4-11db-4f95-babe-c571c6a9bf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:09.254587 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfeea4-11db-4f95-babe-c571c6a9bf0e admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:09.256356 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfeea4-11db-4f95-babe-c571c6a9bf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:09.256356 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cfeea4-11db-4f95-babe-c571c6a9bf0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:09.256356 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cfeea4-11db-4f95-babe-c571c6a9bf0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:52:09.256802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cfeea4-11db-4f95-babe-c571c6a9bf0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:09.258086 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cfeea4-11db-4f95-babe-c571c6a9bf0e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:09.272305 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1161/2312] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:52:09.277986 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0adaf3-a3c1-4735-82c9-8a5300535822 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:09.288025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0adaf3-a3c1-4735-82c9-8a5300535822 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:09.295041 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e4397-2a8a-4b7f-9b1b-23be3c1bc16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:09.295782 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e4397-2a8a-4b7f-9b1b-23be3c1bc16b admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:09.296065 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e4397-2a8a-4b7f-9b1b-23be3c1bc16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:09.296410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e4397-2a8a-4b7f-9b1b-23be3c1bc16b admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:09.297050 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e4397-2a8a-4b7f-9b1b-23be3c1bc16b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:09.300125 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e4397-2a8a-4b7f-9b1b-23be3c1bc16b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:09.417948 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1152/2313] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:52:09 2022] POST /identity/v3/projects => generated 403 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:52:09.419537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cfeea4-11db-4f95-babe-c571c6a9bf0e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:09.437084 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cfeea4-11db-4f95-babe-c571c6a9bf0e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:09.531416 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ec33c-5757-4f9a-bd5e-6b9fff7867dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:09.532167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ec33c-5757-4f9a-bd5e-6b9fff7867dc admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:09.532457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ec33c-5757-4f9a-bd5e-6b9fff7867dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:09.533343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ec33c-5757-4f9a-bd5e-6b9fff7867dc admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:09.534058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ec33c-5757-4f9a-bd5e-6b9fff7867dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:09.536107 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ec33c-5757-4f9a-bd5e-6b9fff7867dc admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:09.663551 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1162/2314] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:52:09 2022] POST /identity/v3/users => generated 302 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:52:09.667882 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e4397-2a8a-4b7f-9b1b-23be3c1bc16b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:09.678818 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e4397-2a8a-4b7f-9b1b-23be3c1bc16b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:09.683903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3cbfc-5f14-4d84-a3b2-7ba2269067c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:09.684697 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3cbfc-5f14-4d84-a3b2-7ba2269067c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:09.684983 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3cbfc-5f14-4d84-a3b2-7ba2269067c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:09.685258 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3cbfc-5f14-4d84-a3b2-7ba2269067c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:09.686000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3cbfc-5f14-4d84-a3b2-7ba2269067c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:09.687694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3cbfc-5f14-4d84-a3b2-7ba2269067c6 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:09.700334 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1153/2315] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:52:09.705602 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ec33c-5757-4f9a-bd5e-6b9fff7867dc admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:09.716414 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ec33c-5757-4f9a-bd5e-6b9fff7867dc admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:09.722326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e10bf-ee73-4a3d-8cb8-2048e2dd8885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:09.723018 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e10bf-ee73-4a3d-8cb8-2048e2dd8885 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:09.723299 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e10bf-ee73-4a3d-8cb8-2048e2dd8885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:09.731876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e10bf-ee73-4a3d-8cb8-2048e2dd8885 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:09.731876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e10bf-ee73-4a3d-8cb8-2048e2dd8885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:09.731876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e10bf-ee73-4a3d-8cb8-2048e2dd8885 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:09.737116 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1163/2316] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:52:09.741940 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3cbfc-5f14-4d84-a3b2-7ba2269067c6 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:09.753450 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3cbfc-5f14-4d84-a3b2-7ba2269067c6 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:09.758516 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044548bb-cebb-4410-9919-9b70d62b04fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:09.759421 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044548bb-cebb-4410-9919-9b70d62b04fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:09.759738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044548bb-cebb-4410-9919-9b70d62b04fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:09.760087 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044548bb-cebb-4410-9919-9b70d62b04fb admin admin] PATH_INFO: `/v3/projects/2912a27cf1c745b8bb78fedd45176016/users/d552e3b525174ce8a1f4d90882d7a1e3/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:09.784997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044548bb-cebb-4410-9919-9b70d62b04fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2912a27cf1c745b8bb78fedd45176016, user_id=d552e3b525174ce8a1f4d90882d7a1e3, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:09.784997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044548bb-cebb-4410-9919-9b70d62b04fb admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:09.968277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1154/2317] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:52:09 2022] PUT /identity/v3/projects/2912a27cf1c745b8bb78fedd45176016/users/d552e3b525174ce8a1f4d90882d7a1e3/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:09.980328 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e8580-98e2-4337-8d34-8753056ac8a0 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:09.980621 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e8580-98e2-4337-8d34-8753056ac8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:09.980905 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e8580-98e2-4337-8d34-8753056ac8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:10.011514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a95e8580-98e2-4337-8d34-8753056ac8a0 None None] MFA Rules not processed for user `d552e3b525174ce8a1f4d90882d7a1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:10.012863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a95e8580-98e2-4337-8d34-8753056ac8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:10.091340 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1164/2318] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:10.096476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044548bb-cebb-4410-9919-9b70d62b04fb admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:10.178350 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044548bb-cebb-4410-9919-9b70d62b04fb admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:10.194122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfc646-df49-48cc-8f50-44fea7302f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:10.194977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfc646-df49-48cc-8f50-44fea7302f58 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:10.195350 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfc646-df49-48cc-8f50-44fea7302f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:10.195705 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfc646-df49-48cc-8f50-44fea7302f58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:10.196830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfc646-df49-48cc-8f50-44fea7302f58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:10.198453 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfc646-df49-48cc-8f50-44fea7302f58 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:10.303925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1155/2319] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:52:10 2022] POST /identity/v3/projects => generated 405 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:52:10.308637 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e8580-98e2-4337-8d34-8753056ac8a0 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:10.320558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e8580-98e2-4337-8d34-8753056ac8a0 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:10.335715 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:10.385995 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b213844-c7cf-4e2f-aaab-37262a850c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:10.389440 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b213844-c7cf-4e2f-aaab-37262a850c64 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:10.389440 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b213844-c7cf-4e2f-aaab-37262a850c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:10.389440 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b213844-c7cf-4e2f-aaab-37262a850c64 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:10.389440 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b213844-c7cf-4e2f-aaab-37262a850c64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:10.390159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b213844-c7cf-4e2f-aaab-37262a850c64 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:10.560265 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1165/2320] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:52:10 2022] POST /identity/v3/users => generated 309 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:52:10.561849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfc646-df49-48cc-8f50-44fea7302f58 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:10.573045 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfc646-df49-48cc-8f50-44fea7302f58 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:10.578306 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d797fdb-a62b-43aa-b17d-bddb52fa77f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:10.578969 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d797fdb-a62b-43aa-b17d-bddb52fa77f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:10.579255 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d797fdb-a62b-43aa-b17d-bddb52fa77f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:10.579944 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d797fdb-a62b-43aa-b17d-bddb52fa77f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:10.580655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d797fdb-a62b-43aa-b17d-bddb52fa77f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:10.582264 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d797fdb-a62b-43aa-b17d-bddb52fa77f9 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:10.602387 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1156/2321] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:52:10.602607 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b213844-c7cf-4e2f-aaab-37262a850c64 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:10.611996 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b213844-c7cf-4e2f-aaab-37262a850c64 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:10.617536 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae8277-bb97-4b01-918f-735078ae1568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:10.618405 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae8277-bb97-4b01-918f-735078ae1568 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:10.621988 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae8277-bb97-4b01-918f-735078ae1568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:10.621988 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae8277-bb97-4b01-918f-735078ae1568 admin admin] PATH_INFO: `/v3/projects/ec9063d71e1e4bea859b16b4e26d34ae/users/99405061e0c140eebfc6e39b0b8573a1/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:10.638641 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae8277-bb97-4b01-918f-735078ae1568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec9063d71e1e4bea859b16b4e26d34ae, user_id=99405061e0c140eebfc6e39b0b8573a1, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:10.641542 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae8277-bb97-4b01-918f-735078ae1568 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:10.734138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1166/2322] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:52:10 2022] PUT /identity/v3/projects/ec9063d71e1e4bea859b16b4e26d34ae/users/99405061e0c140eebfc6e39b0b8573a1/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:10.739457 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d797fdb-a62b-43aa-b17d-bddb52fa77f9 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:10.750982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d797fdb-a62b-43aa-b17d-bddb52fa77f9 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:10.806822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:10.813901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c9ce4-a998-47af-8b48-ab4c120e299a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:10.814594 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c9ce4-a998-47af-8b48-ab4c120e299a admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:10.814875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c9ce4-a998-47af-8b48-ab4c120e299a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:10.815151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c9ce4-a998-47af-8b48-ab4c120e299a admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:10.815927 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c9ce4-a998-47af-8b48-ab4c120e299a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:10.817541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c9ce4-a998-47af-8b48-ab4c120e299a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:10.828684 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1157/2323] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:52:10.833147 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae8277-bb97-4b01-918f-735078ae1568 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:10.844024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae8277-bb97-4b01-918f-735078ae1568 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:10.850115 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a091eb-3b41-4669-9169-70e4d1faa24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:10.851033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a091eb-3b41-4669-9169-70e4d1faa24a admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:10.851327 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a091eb-3b41-4669-9169-70e4d1faa24a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:10.851608 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a091eb-3b41-4669-9169-70e4d1faa24a admin admin] PATH_INFO: `/v3/projects/ec9063d71e1e4bea859b16b4e26d34ae/users/99405061e0c140eebfc6e39b0b8573a1/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:10.856922 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a091eb-3b41-4669-9169-70e4d1faa24a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec9063d71e1e4bea859b16b4e26d34ae, user_id=99405061e0c140eebfc6e39b0b8573a1, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:10.857092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a091eb-3b41-4669-9169-70e4d1faa24a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:10.870188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1167/2324] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:52:10 2022] PUT /identity/v3/projects/ec9063d71e1e4bea859b16b4e26d34ae/users/99405061e0c140eebfc6e39b0b8573a1/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:10.881694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e36ac4-933e-47dc-ad53-0e9b79f0a7a6 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:10.881984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e36ac4-933e-47dc-ad53-0e9b79f0a7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:10.882274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e36ac4-933e-47dc-ad53-0e9b79f0a7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:10.914559 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c7e36ac4-933e-47dc-ad53-0e9b79f0a7a6 None None] MFA Rules not processed for user `99405061e0c140eebfc6e39b0b8573a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:10.915907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c7e36ac4-933e-47dc-ad53-0e9b79f0a7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:10.992646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1158/2325] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:10 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:11.004387 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a091eb-3b41-4669-9169-70e4d1faa24a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:11.006126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-76a091eb-3b41-4669-9169-70e4d1faa24a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:11.073719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a091eb-3b41-4669-9169-70e4d1faa24a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:11.077918 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664b414-a768-4a4b-b6b5-dac10ba93827 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:11.078575 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664b414-a768-4a4b-b6b5-dac10ba93827 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:11.078863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664b414-a768-4a4b-b6b5-dac10ba93827 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:11.079144 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664b414-a768-4a4b-b6b5-dac10ba93827 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:11.089968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0664b414-a768-4a4b-b6b5-dac10ba93827 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:11.091800 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0664b414-a768-4a4b-b6b5-dac10ba93827 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:11.102220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1168/2326] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:52:11 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:52:11.246818 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e36ac4-933e-47dc-ad53-0e9b79f0a7a6 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:11.319660 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e36ac4-933e-47dc-ad53-0e9b79f0a7a6 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:11.324726 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92721515-f126-44b7-8cb5-b2a6bdbd4139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:11.325610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92721515-f126-44b7-8cb5-b2a6bdbd4139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:11.325920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92721515-f126-44b7-8cb5-b2a6bdbd4139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:11.326204 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92721515-f126-44b7-8cb5-b2a6bdbd4139 admin admin] PATH_INFO: `/v3/users/d552e3b525174ce8a1f4d90882d7a1e3` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:11.327694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92721515-f126-44b7-8cb5-b2a6bdbd4139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d552e3b525174ce8a1f4d90882d7a1e3)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:11.329729 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92721515-f126-44b7-8cb5-b2a6bdbd4139 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:11.498502 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-92721515-f126-44b7-8cb5-b2a6bdbd4139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd552e3b525174ce8a1f4d90882d7a1e3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:11.499183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-92721515-f126-44b7-8cb5-b2a6bdbd4139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd552e3b525174ce8a1f4d90882d7a1e3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:11.500025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-92721515-f126-44b7-8cb5-b2a6bdbd4139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd552e3b525174ce8a1f4d90882d7a1e3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:11.513990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-92721515-f126-44b7-8cb5-b2a6bdbd4139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd552e3b525174ce8a1f4d90882d7a1e3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:11.521737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-92721515-f126-44b7-8cb5-b2a6bdbd4139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd552e3b525174ce8a1f4d90882d7a1e3'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:11.526494 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:11.615379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1159/2327] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:52:11 2022] DELETE /identity/v3/users/d552e3b525174ce8a1f4d90882d7a1e3 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:11.620765 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff11c3-69f9-49f2-b7ca-35f3766e075e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:11.621197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff11c3-69f9-49f2-b7ca-35f3766e075e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:11.621737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff11c3-69f9-49f2-b7ca-35f3766e075e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:11.643005 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e8ff11c3-69f9-49f2-b7ca-35f3766e075e None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:11.644502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e8ff11c3-69f9-49f2-b7ca-35f3766e075e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:11.710029 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1169/2328] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:52:11.720101 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92721515-f126-44b7-8cb5-b2a6bdbd4139 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:11.722085 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-92721515-f126-44b7-8cb5-b2a6bdbd4139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:11.790804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92721515-f126-44b7-8cb5-b2a6bdbd4139 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:11.794803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8e933-a2d8-4264-8200-a1817f091f15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:11.795480 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8e933-a2d8-4264-8200-a1817f091f15 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:11.795794 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8e933-a2d8-4264-8200-a1817f091f15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:11.796071 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8e933-a2d8-4264-8200-a1817f091f15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:11.807730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8e933-a2d8-4264-8200-a1817f091f15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:11.809190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8e933-a2d8-4264-8200-a1817f091f15 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:11.821374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1160/2329] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:52:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:52:12.353658 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff11c3-69f9-49f2-b7ca-35f3766e075e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:12.355354 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e8ff11c3-69f9-49f2-b7ca-35f3766e075e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:12.368812 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff11c3-69f9-49f2-b7ca-35f3766e075e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:12.375387 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88a048-171f-4deb-83c9-61de2db172c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:12.376518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88a048-171f-4deb-83c9-61de2db172c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:12.376843 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88a048-171f-4deb-83c9-61de2db172c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:12.377182 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88a048-171f-4deb-83c9-61de2db172c0 admin admin] PATH_INFO: `/v3/projects/2912a27cf1c745b8bb78fedd45176016` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:12.378578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88a048-171f-4deb-83c9-61de2db172c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2912a27cf1c745b8bb78fedd45176016)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:12.380788 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88a048-171f-4deb-83c9-61de2db172c0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:12.389971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:12.526221 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fa88a048-171f-4deb-83c9-61de2db172c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2912a27cf1c745b8bb78fedd45176016', 'request_id': 'req-fa88a048-171f-4deb-83c9-61de2db172c0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:12.535101 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fa88a048-171f-4deb-83c9-61de2db172c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2912a27cf1c745b8bb78fedd45176016', 'request_id': 'req-fa88a048-171f-4deb-83c9-61de2db172c0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:12.545423 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fa88a048-171f-4deb-83c9-61de2db172c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2912a27cf1c745b8bb78fedd45176016', 'request_id': 'req-fa88a048-171f-4deb-83c9-61de2db172c0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:12.635779 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fa88a048-171f-4deb-83c9-61de2db172c0 admin admin] The token cache is being invalidate because project 2912a27cf1c745b8bb78fedd45176016 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:52:12.636116 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fa88a048-171f-4deb-83c9-61de2db172c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:12.636786 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fa88a048-171f-4deb-83c9-61de2db172c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:12.638621 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1170/2330] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:52:12 2022] DELETE /identity/v3/projects/2912a27cf1c745b8bb78fedd45176016 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:12.643516 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8e933-a2d8-4264-8200-a1817f091f15 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:12.660629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8e933-a2d8-4264-8200-a1817f091f15 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:12.729772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270018a-8125-46f0-8f1c-cd64d97e56f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:12.730559 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270018a-8125-46f0-8f1c-cd64d97e56f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:12.730842 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270018a-8125-46f0-8f1c-cd64d97e56f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:12.731110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270018a-8125-46f0-8f1c-cd64d97e56f4 admin admin] PATH_INFO: `/v3/users/99405061e0c140eebfc6e39b0b8573a1` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:12.732333 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270018a-8125-46f0-8f1c-cd64d97e56f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99405061e0c140eebfc6e39b0b8573a1)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:12.734335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270018a-8125-46f0-8f1c-cd64d97e56f4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:12.998299 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c270018a-8125-46f0-8f1c-cd64d97e56f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99405061e0c140eebfc6e39b0b8573a1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:12.999015 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c270018a-8125-46f0-8f1c-cd64d97e56f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99405061e0c140eebfc6e39b0b8573a1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:12.999633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c270018a-8125-46f0-8f1c-cd64d97e56f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99405061e0c140eebfc6e39b0b8573a1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:13.015070 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c270018a-8125-46f0-8f1c-cd64d97e56f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99405061e0c140eebfc6e39b0b8573a1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:13.022443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c270018a-8125-46f0-8f1c-cd64d97e56f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99405061e0c140eebfc6e39b0b8573a1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:13.099360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1161/2331] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:52:12 2022] DELETE /identity/v3/users/99405061e0c140eebfc6e39b0b8573a1 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:13.873883 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88a048-171f-4deb-83c9-61de2db172c0 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:13.875351 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fa88a048-171f-4deb-83c9-61de2db172c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:13.890048 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88a048-171f-4deb-83c9-61de2db172c0 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:13.957642 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de161dd7-7cd2-436b-87bc-e4a16c69a3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:13.958555 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de161dd7-7cd2-436b-87bc-e4a16c69a3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:13.958836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de161dd7-7cd2-436b-87bc-e4a16c69a3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:13.959120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de161dd7-7cd2-436b-87bc-e4a16c69a3e8 admin admin] PATH_INFO: `/v3/projects/ec9063d71e1e4bea859b16b4e26d34ae` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:13.960417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de161dd7-7cd2-436b-87bc-e4a16c69a3e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec9063d71e1e4bea859b16b4e26d34ae)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:13.962413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de161dd7-7cd2-436b-87bc-e4a16c69a3e8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:14.084548 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-de161dd7-7cd2-436b-87bc-e4a16c69a3e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec9063d71e1e4bea859b16b4e26d34ae', 'request_id': 'req-de161dd7-7cd2-436b-87bc-e4a16c69a3e8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:14.090076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-de161dd7-7cd2-436b-87bc-e4a16c69a3e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec9063d71e1e4bea859b16b4e26d34ae', 'request_id': 'req-de161dd7-7cd2-436b-87bc-e4a16c69a3e8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:14.097146 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-de161dd7-7cd2-436b-87bc-e4a16c69a3e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec9063d71e1e4bea859b16b4e26d34ae', 'request_id': 'req-de161dd7-7cd2-436b-87bc-e4a16c69a3e8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:14.176804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-de161dd7-7cd2-436b-87bc-e4a16c69a3e8 admin admin] The token cache is being invalidate because project ec9063d71e1e4bea859b16b4e26d34ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:52:14.177008 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-de161dd7-7cd2-436b-87bc-e4a16c69a3e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:14.177685 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-de161dd7-7cd2-436b-87bc-e4a16c69a3e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:14.179275 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1171/2332] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:52:13 2022] DELETE /identity/v3/projects/ec9063d71e1e4bea859b16b4e26d34ae => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:14.184405 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783b391-a323-426b-ac61-18fccbd2bfc4 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:14.184941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783b391-a323-426b-ac61-18fccbd2bfc4 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:14.185055 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783b391-a323-426b-ac61-18fccbd2bfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:14.207430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c783b391-a323-426b-ac61-18fccbd2bfc4 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:14.209604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c783b391-a323-426b-ac61-18fccbd2bfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:14.228502 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:14.277848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1162/2333] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:14.288241 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2965b-8acb-464e-a4e4-2ef632238a10 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:14.288533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2965b-8acb-464e-a4e4-2ef632238a10 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:14.288813 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2965b-8acb-464e-a4e4-2ef632238a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:14.308770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-9da2965b-8acb-464e-a4e4-2ef632238a10 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:14.309997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9da2965b-8acb-464e-a4e4-2ef632238a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:14.318892 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1172/2334] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:14.323719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783b391-a323-426b-ac61-18fccbd2bfc4 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:14.334055 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783b391-a323-426b-ac61-18fccbd2bfc4 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:14.338875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e227b44-8c46-4f3a-8ac3-42a186790174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:14.339558 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e227b44-8c46-4f3a-8ac3-42a186790174 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:14.339874 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e227b44-8c46-4f3a-8ac3-42a186790174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:14.340151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e227b44-8c46-4f3a-8ac3-42a186790174 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:14.340890 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e227b44-8c46-4f3a-8ac3-42a186790174 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:52:14.341199 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e227b44-8c46-4f3a-8ac3-42a186790174 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:14.342811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e227b44-8c46-4f3a-8ac3-42a186790174 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:14.356114 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1163/2335] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:52:14.360508 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da2965b-8acb-464e-a4e4-2ef632238a10 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:14.371306 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da2965b-8acb-464e-a4e4-2ef632238a10 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:14.376949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4a6c0-0ecc-48c2-b226-d386ae986924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:14.378085 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4a6c0-0ecc-48c2-b226-d386ae986924 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:14.378267 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4a6c0-0ecc-48c2-b226-d386ae986924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:14.380671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4a6c0-0ecc-48c2-b226-d386ae986924 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:14.381568 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4a6c0-0ecc-48c2-b226-d386ae986924 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:14.383531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4a6c0-0ecc-48c2-b226-d386ae986924 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:14.455935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:14.463456 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1173/2336] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:52:14 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:52:14.467648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e227b44-8c46-4f3a-8ac3-42a186790174 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:14.479263 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e227b44-8c46-4f3a-8ac3-42a186790174 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:14.550440 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157846a3-06d8-4cac-8c96-4f6116c1c044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:14.555585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157846a3-06d8-4cac-8c96-4f6116c1c044 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:14.555585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157846a3-06d8-4cac-8c96-4f6116c1c044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:14.555585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157846a3-06d8-4cac-8c96-4f6116c1c044 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:14.555585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157846a3-06d8-4cac-8c96-4f6116c1c044 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:14.555585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157846a3-06d8-4cac-8c96-4f6116c1c044 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:14.638850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1164/2337] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:52:14 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:52:14.643597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4a6c0-0ecc-48c2-b226-d386ae986924 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:14.654161 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4a6c0-0ecc-48c2-b226-d386ae986924 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:14.659858 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d9894-d007-4b87-9704-5b991f2d0652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:14.660580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d9894-d007-4b87-9704-5b991f2d0652 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:14.660864 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d9894-d007-4b87-9704-5b991f2d0652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:14.661143 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d9894-d007-4b87-9704-5b991f2d0652 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:14.661898 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d9894-d007-4b87-9704-5b991f2d0652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:14.663555 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d9894-d007-4b87-9704-5b991f2d0652 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:14.676386 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1174/2338] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:52:14.680592 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157846a3-06d8-4cac-8c96-4f6116c1c044 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:14.692108 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157846a3-06d8-4cac-8c96-4f6116c1c044 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:14.697171 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ee5e7-5e7d-482a-a9d6-114c4a72e046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:14.697842 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ee5e7-5e7d-482a-a9d6-114c4a72e046 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:14.698124 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ee5e7-5e7d-482a-a9d6-114c4a72e046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:14.698398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ee5e7-5e7d-482a-a9d6-114c4a72e046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:14.699221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ee5e7-5e7d-482a-a9d6-114c4a72e046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:14.700804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ee5e7-5e7d-482a-a9d6-114c4a72e046 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:14.714317 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1165/2339] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:52:14.718901 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d9894-d007-4b87-9704-5b991f2d0652 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:14.729731 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d9894-d007-4b87-9704-5b991f2d0652 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:14.735351 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca179c-1662-4689-9e17-13215d895f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:14.736351 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca179c-1662-4689-9e17-13215d895f5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:14.736654 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca179c-1662-4689-9e17-13215d895f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:14.736928 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca179c-1662-4689-9e17-13215d895f5f admin admin] PATH_INFO: `/v3/projects/2efea3a0062f416583523c6f63a98586/users/36bfe5d7da26481e898d853b54102753/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:14.753674 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca179c-1662-4689-9e17-13215d895f5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2efea3a0062f416583523c6f63a98586, user_id=36bfe5d7da26481e898d853b54102753, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:14.756860 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca179c-1662-4689-9e17-13215d895f5f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:14.867608 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1175/2340] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:52:14 2022] PUT /identity/v3/projects/2efea3a0062f416583523c6f63a98586/users/36bfe5d7da26481e898d853b54102753/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:14.880093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b8e50-97bb-4a85-8376-47ac22626780 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:14.881089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b8e50-97bb-4a85-8376-47ac22626780 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:14.881089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b8e50-97bb-4a85-8376-47ac22626780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:14.920009 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-ab0b8e50-97bb-4a85-8376-47ac22626780 None None] MFA Rules not processed for user `36bfe5d7da26481e898d853b54102753`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:14.921325 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ab0b8e50-97bb-4a85-8376-47ac22626780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:14.992597 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:15.015297 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1166/2341] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:15.020764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca179c-1662-4689-9e17-13215d895f5f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:15.144187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca179c-1662-4689-9e17-13215d895f5f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:15.148866 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13c798-6316-482d-8574-304e64d3b690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:15.149520 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13c798-6316-482d-8574-304e64d3b690 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:15.149802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13c798-6316-482d-8574-304e64d3b690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:15.150075 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13c798-6316-482d-8574-304e64d3b690 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:15.150761 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13c798-6316-482d-8574-304e64d3b690 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:15.152692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13c798-6316-482d-8574-304e64d3b690 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:15.255157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1176/2342] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:52:15 2022] POST /identity/v3/projects => generated 379 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:52:15.256321 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b8e50-97bb-4a85-8376-47ac22626780 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:15.267531 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b8e50-97bb-4a85-8376-47ac22626780 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:15.331559 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abab3b1-3b6e-4554-94d6-790d4e248c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:15.332293 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abab3b1-3b6e-4554-94d6-790d4e248c61 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:15.332574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abab3b1-3b6e-4554-94d6-790d4e248c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:15.332846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abab3b1-3b6e-4554-94d6-790d4e248c61 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:15.334348 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abab3b1-3b6e-4554-94d6-790d4e248c61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:15.335728 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abab3b1-3b6e-4554-94d6-790d4e248c61 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:15.415738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1167/2343] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:52:15 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:52:15.420032 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13c798-6316-482d-8574-304e64d3b690 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:15.432032 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13c798-6316-482d-8574-304e64d3b690 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:15.436744 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d833c-215a-492b-bfec-c675a5be6810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:15.437415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d833c-215a-492b-bfec-c675a5be6810 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:15.437701 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d833c-215a-492b-bfec-c675a5be6810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:15.438092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d833c-215a-492b-bfec-c675a5be6810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:15.438827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d833c-215a-492b-bfec-c675a5be6810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:15.441085 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d833c-215a-492b-bfec-c675a5be6810 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:15.453338 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1177/2344] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:52:15.457856 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abab3b1-3b6e-4554-94d6-790d4e248c61 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:15.475106 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abab3b1-3b6e-4554-94d6-790d4e248c61 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:15.482814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a47db4-863c-4ee9-b85b-db89ffc62ade None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:15.483399 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a47db4-863c-4ee9-b85b-db89ffc62ade None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:15.484793 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a47db4-863c-4ee9-b85b-db89ffc62ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:15.485151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4231208-39aa-4831-9858-66e1dc49e036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:15.486870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4231208-39aa-4831-9858-66e1dc49e036 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:15.487483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4231208-39aa-4831-9858-66e1dc49e036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:15.488049 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4231208-39aa-4831-9858-66e1dc49e036 admin admin] PATH_INFO: `/v3/projects/bd21a4929940455e93104f80651a0a18/users/e649def3552f49669e79764ea2a464ec/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:15.514280 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4231208-39aa-4831-9858-66e1dc49e036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd21a4929940455e93104f80651a0a18, user_id=e649def3552f49669e79764ea2a464ec, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:15.516449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a0a47db4-863c-4ee9-b85b-db89ffc62ade None None] MFA Rules not processed for user `2b03f8b8c51e472bbd4cf0afc9c6b632`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:15.516967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4231208-39aa-4831-9858-66e1dc49e036 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:15.517846 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a0a47db4-863c-4ee9-b85b-db89ffc62ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:15.550215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:15.575452 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1178/2345] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Jul 18 08:52:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:15.605800 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1168/2346] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:52:15 2022] PUT /identity/v3/projects/bd21a4929940455e93104f80651a0a18/users/e649def3552f49669e79764ea2a464ec/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:52:15.612987 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a47db4-863c-4ee9-b85b-db89ffc62ade None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:15.625983 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a47db4-863c-4ee9-b85b-db89ffc62ade None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:15.689076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bceb01-77aa-4294-88bc-2049de4708a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:15.689777 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bceb01-77aa-4294-88bc-2049de4708a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:15.690060 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bceb01-77aa-4294-88bc-2049de4708a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:15.690344 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bceb01-77aa-4294-88bc-2049de4708a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:15.691088 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bceb01-77aa-4294-88bc-2049de4708a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:15.692814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bceb01-77aa-4294-88bc-2049de4708a0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:15.708534 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1179/2347] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:52:15.710505 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4231208-39aa-4831-9858-66e1dc49e036 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:15.722853 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4231208-39aa-4831-9858-66e1dc49e036 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:15.729570 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2114fc-bc99-4267-b02c-725935f7c8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:15.730821 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2114fc-bc99-4267-b02c-725935f7c8f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:15.731104 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2114fc-bc99-4267-b02c-725935f7c8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:15.731391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2114fc-bc99-4267-b02c-725935f7c8f7 admin admin] PATH_INFO: `/v3/projects/bd21a4929940455e93104f80651a0a18/users/e649def3552f49669e79764ea2a464ec/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:15.733797 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2114fc-bc99-4267-b02c-725935f7c8f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd21a4929940455e93104f80651a0a18, user_id=e649def3552f49669e79764ea2a464ec, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:15.740602 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2114fc-bc99-4267-b02c-725935f7c8f7 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:15.747943 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bceb01-77aa-4294-88bc-2049de4708a0 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:15.750881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1169/2348] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:52:15 2022] PUT /identity/v3/projects/bd21a4929940455e93104f80651a0a18/users/e649def3552f49669e79764ea2a464ec/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:15.752438 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-06bceb01-77aa-4294-88bc-2049de4708a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:15.761918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ecbc7-8451-4a4a-b35f-b8e6dc2183be None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:15.762138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ecbc7-8451-4a4a-b35f-b8e6dc2183be None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:15.762405 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ecbc7-8451-4a4a-b35f-b8e6dc2183be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:15.794517 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-395ecbc7-8451-4a4a-b35f-b8e6dc2183be None None] MFA Rules not processed for user `e649def3552f49669e79764ea2a464ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:15.795776 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-395ecbc7-8451-4a4a-b35f-b8e6dc2183be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:15.872708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bceb01-77aa-4294-88bc-2049de4708a0 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:15.901933 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1170/2349] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:15.922435 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ecbc7-8451-4a4a-b35f-b8e6dc2183be None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:15.926213 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-395ecbc7-8451-4a4a-b35f-b8e6dc2183be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:15.959947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:15.964133 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a0f9f5-f2c2-4a89-82ea-9119d1b73f48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09a5c2e804504038817cad76544dce5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:15.964838 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0f9f5-f2c2-4a89-82ea-9119d1b73f48 service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:15.965126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0f9f5-f2c2-4a89-82ea-9119d1b73f48 service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:15.965409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0f9f5-f2c2-4a89-82ea-9119d1b73f48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:15.969953 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:16.021188 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ecbc7-8451-4a4a-b35f-b8e6dc2183be None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:16.025411 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182fa71-834c-49a5-ae1d-09b6ff7d09a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:16.026085 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182fa71-834c-49a5-ae1d-09b6ff7d09a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:16.026371 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182fa71-834c-49a5-ae1d-09b6ff7d09a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:16.026644 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182fa71-834c-49a5-ae1d-09b6ff7d09a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:16.031286 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a0f9f5-f2c2-4a89-82ea-9119d1b73f48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:16.034116 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a0f9f5-f2c2-4a89-82ea-9119d1b73f48 service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:16.040000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182fa71-834c-49a5-ae1d-09b6ff7d09a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:16.042055 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182fa71-834c-49a5-ae1d-09b6ff7d09a1 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:16.044238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1180/2350] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Jul 18 08:52:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 08:52:16.057567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1171/2351] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:52:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:52:16.327432 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a0f9f5-f2c2-4a89-82ea-9119d1b73f48 service swift] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:16.341838 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a0f9f5-f2c2-4a89-82ea-9119d1b73f48 service swift] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:16.348551 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f8eb8-b282-4f88-bf22-882ca4b368ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:16.348551 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f8eb8-b282-4f88-bf22-882ca4b368ed service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:16.348551 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f8eb8-b282-4f88-bf22-882ca4b368ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:16.349059 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f8eb8-b282-4f88-bf22-882ca4b368ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:16.449620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f8eb8-b282-4f88-bf22-882ca4b368ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:16.451180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f8eb8-b282-4f88-bf22-882ca4b368ed service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:16.471044 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1181/2352] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:52:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:52:18.711437 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182fa71-834c-49a5-ae1d-09b6ff7d09a1 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:18.722465 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7182fa71-834c-49a5-ae1d-09b6ff7d09a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:19.009186 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182fa71-834c-49a5-ae1d-09b6ff7d09a1 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:19.019263 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c775636-a3ac-4f10-baf0-c1ed6f9683e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:19.020030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c775636-a3ac-4f10-baf0-c1ed6f9683e0 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:19.020861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c775636-a3ac-4f10-baf0-c1ed6f9683e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:19.021158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c775636-a3ac-4f10-baf0-c1ed6f9683e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:19.024414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2c775636-a3ac-4f10-baf0-c1ed6f9683e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:19.052935 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c775636-a3ac-4f10-baf0-c1ed6f9683e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:19.055568 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c775636-a3ac-4f10-baf0-c1ed6f9683e0 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:19.071891 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1172/2353] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:52:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:52:26.460858 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=53 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:29.066309 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=58 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:31.140753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f8eb8-b282-4f88-bf22-882ca4b368ed service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:31.232493 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f8eb8-b282-4f88-bf22-882ca4b368ed service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:31.248905 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f701dfa-c89e-40d6-a7f5-196d5e135c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:31.249908 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f701dfa-c89e-40d6-a7f5-196d5e135c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:31.250192 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f701dfa-c89e-40d6-a7f5-196d5e135c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:31.250463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f701dfa-c89e-40d6-a7f5-196d5e135c02 admin admin] PATH_INFO: `/v3/users/36bfe5d7da26481e898d853b54102753` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:31.252749 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f701dfa-c89e-40d6-a7f5-196d5e135c02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36bfe5d7da26481e898d853b54102753)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:31.255344 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f701dfa-c89e-40d6-a7f5-196d5e135c02 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:31.497682 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6f701dfa-c89e-40d6-a7f5-196d5e135c02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36bfe5d7da26481e898d853b54102753'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:31.498365 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6f701dfa-c89e-40d6-a7f5-196d5e135c02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36bfe5d7da26481e898d853b54102753'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:31.498971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6f701dfa-c89e-40d6-a7f5-196d5e135c02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36bfe5d7da26481e898d853b54102753'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:31.511967 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6f701dfa-c89e-40d6-a7f5-196d5e135c02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36bfe5d7da26481e898d853b54102753'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:31.519444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6f701dfa-c89e-40d6-a7f5-196d5e135c02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36bfe5d7da26481e898d853b54102753'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:31.602751 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1182/2354] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:52:31 2022] DELETE /identity/v3/users/36bfe5d7da26481e898d853b54102753 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:31.607048 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead2947-4192-406a-8742-03cca7089873 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:31.607472 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead2947-4192-406a-8742-03cca7089873 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:31.607605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead2947-4192-406a-8742-03cca7089873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:31.637880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0ead2947-4192-406a-8742-03cca7089873 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:31.639137 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0ead2947-4192-406a-8742-03cca7089873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:31.710803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1173/2355] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:31.723395 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f701dfa-c89e-40d6-a7f5-196d5e135c02 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:31.724954 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6f701dfa-c89e-40d6-a7f5-196d5e135c02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:31.864145 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f701dfa-c89e-40d6-a7f5-196d5e135c02 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:31.868498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a04cdf-bf4f-453a-aa02-387aeb5ccd6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:31.869282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a04cdf-bf4f-453a-aa02-387aeb5ccd6f service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:31.869577 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a04cdf-bf4f-453a-aa02-387aeb5ccd6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:31.869862 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a04cdf-bf4f-453a-aa02-387aeb5ccd6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:31.884360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a04cdf-bf4f-453a-aa02-387aeb5ccd6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:31.885890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a04cdf-bf4f-453a-aa02-387aeb5ccd6f service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:31.897412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1183/2356] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:52:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:52:32.368142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead2947-4192-406a-8742-03cca7089873 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:32.370710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0ead2947-4192-406a-8742-03cca7089873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:32.396299 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead2947-4192-406a-8742-03cca7089873 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:32.402316 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70d7fc-a509-4624-8f0e-b6f44bdd5804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:32.403164 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70d7fc-a509-4624-8f0e-b6f44bdd5804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:32.403444 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70d7fc-a509-4624-8f0e-b6f44bdd5804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:32.403788 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70d7fc-a509-4624-8f0e-b6f44bdd5804 admin admin] PATH_INFO: `/v3/projects/2efea3a0062f416583523c6f63a98586` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:32.405038 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70d7fc-a509-4624-8f0e-b6f44bdd5804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2efea3a0062f416583523c6f63a98586)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:32.407549 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70d7fc-a509-4624-8f0e-b6f44bdd5804 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:32.538022 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0c70d7fc-a509-4624-8f0e-b6f44bdd5804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2efea3a0062f416583523c6f63a98586', 'request_id': 'req-0c70d7fc-a509-4624-8f0e-b6f44bdd5804'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:32.546135 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0c70d7fc-a509-4624-8f0e-b6f44bdd5804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2efea3a0062f416583523c6f63a98586', 'request_id': 'req-0c70d7fc-a509-4624-8f0e-b6f44bdd5804'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:32.553821 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0c70d7fc-a509-4624-8f0e-b6f44bdd5804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2efea3a0062f416583523c6f63a98586', 'request_id': 'req-0c70d7fc-a509-4624-8f0e-b6f44bdd5804'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:32.635741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0c70d7fc-a509-4624-8f0e-b6f44bdd5804 admin admin] The token cache is being invalidate because project 2efea3a0062f416583523c6f63a98586 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:52:32.636477 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0c70d7fc-a509-4624-8f0e-b6f44bdd5804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:32.636771 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-0c70d7fc-a509-4624-8f0e-b6f44bdd5804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:32.638616 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1174/2357] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:52:32 2022] DELETE /identity/v3/projects/2efea3a0062f416583523c6f63a98586 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:32.643286 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a04cdf-bf4f-453a-aa02-387aeb5ccd6f service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:32.648504 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:32.654479 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a04cdf-bf4f-453a-aa02-387aeb5ccd6f service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:32.722569 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde30d0a-eac8-4760-af31-f1997cdb0d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:32.723470 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde30d0a-eac8-4760-af31-f1997cdb0d7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:32.723792 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde30d0a-eac8-4760-af31-f1997cdb0d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:32.724075 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde30d0a-eac8-4760-af31-f1997cdb0d7f admin admin] PATH_INFO: `/v3/users/e649def3552f49669e79764ea2a464ec` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:32.725428 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde30d0a-eac8-4760-af31-f1997cdb0d7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e649def3552f49669e79764ea2a464ec)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:32.727511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde30d0a-eac8-4760-af31-f1997cdb0d7f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:32.972941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bde30d0a-eac8-4760-af31-f1997cdb0d7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e649def3552f49669e79764ea2a464ec'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:32.973672 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bde30d0a-eac8-4760-af31-f1997cdb0d7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e649def3552f49669e79764ea2a464ec'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:32.974351 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bde30d0a-eac8-4760-af31-f1997cdb0d7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e649def3552f49669e79764ea2a464ec'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:32.989609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bde30d0a-eac8-4760-af31-f1997cdb0d7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e649def3552f49669e79764ea2a464ec'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:33.000732 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bde30d0a-eac8-4760-af31-f1997cdb0d7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e649def3552f49669e79764ea2a464ec'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:33.121853 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1184/2358] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:52:32 2022] DELETE /identity/v3/users/e649def3552f49669e79764ea2a464ec => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:52:33.794610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70d7fc-a509-4624-8f0e-b6f44bdd5804 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:33.801194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0c70d7fc-a509-4624-8f0e-b6f44bdd5804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:33.812697 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70d7fc-a509-4624-8f0e-b6f44bdd5804 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:33.882025 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9cb96-fbd0-480d-a19f-dd0a46919d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:33.882913 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9cb96-fbd0-480d-a19f-dd0a46919d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:33.883196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9cb96-fbd0-480d-a19f-dd0a46919d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:33.883469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9cb96-fbd0-480d-a19f-dd0a46919d04 admin admin] PATH_INFO: `/v3/projects/bd21a4929940455e93104f80651a0a18` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:33.884781 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9cb96-fbd0-480d-a19f-dd0a46919d04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd21a4929940455e93104f80651a0a18)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:33.886840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9cb96-fbd0-480d-a19f-dd0a46919d04 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:33.911485 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:33.998237 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-12f9cb96-fbd0-480d-a19f-dd0a46919d04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd21a4929940455e93104f80651a0a18', 'request_id': 'req-12f9cb96-fbd0-480d-a19f-dd0a46919d04'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:34.004232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-12f9cb96-fbd0-480d-a19f-dd0a46919d04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd21a4929940455e93104f80651a0a18', 'request_id': 'req-12f9cb96-fbd0-480d-a19f-dd0a46919d04'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:34.010833 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-12f9cb96-fbd0-480d-a19f-dd0a46919d04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd21a4929940455e93104f80651a0a18', 'request_id': 'req-12f9cb96-fbd0-480d-a19f-dd0a46919d04'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:34.113008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-12f9cb96-fbd0-480d-a19f-dd0a46919d04 admin admin] The token cache is being invalidate because project bd21a4929940455e93104f80651a0a18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:52:34.113215 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-12f9cb96-fbd0-480d-a19f-dd0a46919d04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:34.113938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-12f9cb96-fbd0-480d-a19f-dd0a46919d04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:34.115565 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1175/2359] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:52:33 2022] DELETE /identity/v3/projects/bd21a4929940455e93104f80651a0a18 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:34.120673 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05feeb-2a3b-4f83-935b-547b1005e328 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:34.121195 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05feeb-2a3b-4f83-935b-547b1005e328 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:34.121312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05feeb-2a3b-4f83-935b-547b1005e328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:34.173376 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-6a05feeb-2a3b-4f83-935b-547b1005e328 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:34.174638 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6a05feeb-2a3b-4f83-935b-547b1005e328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:34.234800 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:34.246789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1185/2360] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:34.254991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c4394-f919-4c63-898f-666cd1c1342d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:34.255271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c4394-f919-4c63-898f-666cd1c1342d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:34.255548 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c4394-f919-4c63-898f-666cd1c1342d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:34.274905 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-743c4394-f919-4c63-898f-666cd1c1342d None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:34.276328 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-743c4394-f919-4c63-898f-666cd1c1342d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:34.284633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1176/2361] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:52:34.289766 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a05feeb-2a3b-4f83-935b-547b1005e328 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:34.306703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a05feeb-2a3b-4f83-935b-547b1005e328 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:34.311989 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5ec22-3b95-430c-bbc9-06565d2c5442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:34.312658 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5ec22-3b95-430c-bbc9-06565d2c5442 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:34.312948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5ec22-3b95-430c-bbc9-06565d2c5442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:34.313846 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5ec22-3b95-430c-bbc9-06565d2c5442 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:34.314638 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5ec22-3b95-430c-bbc9-06565d2c5442 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:52:34.314958 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5ec22-3b95-430c-bbc9-06565d2c5442 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:34.316758 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5ec22-3b95-430c-bbc9-06565d2c5442 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:34.331757 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1186/2362] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:52:34.336433 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c4394-f919-4c63-898f-666cd1c1342d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:34.347936 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c4394-f919-4c63-898f-666cd1c1342d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:34.353000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af33e0-3ccb-44d4-9a33-51b7074efa38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:34.353745 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af33e0-3ccb-44d4-9a33-51b7074efa38 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:34.354074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af33e0-3ccb-44d4-9a33-51b7074efa38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:34.354347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af33e0-3ccb-44d4-9a33-51b7074efa38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:34.355073 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af33e0-3ccb-44d4-9a33-51b7074efa38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:34.357020 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af33e0-3ccb-44d4-9a33-51b7074efa38 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:34.441241 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1177/2363] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:52:34 2022] POST /identity/v3/projects => generated 361 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:52:34.447323 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5ec22-3b95-430c-bbc9-06565d2c5442 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:34.469042 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5ec22-3b95-430c-bbc9-06565d2c5442 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:34.556981 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891ab01-121c-4336-8d3a-a0779b4b0c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:34.557601 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891ab01-121c-4336-8d3a-a0779b4b0c72 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:34.557896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891ab01-121c-4336-8d3a-a0779b4b0c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:34.558186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891ab01-121c-4336-8d3a-a0779b4b0c72 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:34.558917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891ab01-121c-4336-8d3a-a0779b4b0c72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:34.562535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891ab01-121c-4336-8d3a-a0779b4b0c72 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:34.660573 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1187/2364] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:52:34 2022] POST /identity/v3/users => generated 281 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:52:34.664991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af33e0-3ccb-44d4-9a33-51b7074efa38 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:34.680052 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af33e0-3ccb-44d4-9a33-51b7074efa38 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:34.690069 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637dcdf6-e0d4-44a9-9617-6b93fca92806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:34.694156 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637dcdf6-e0d4-44a9-9617-6b93fca92806 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:34.694156 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637dcdf6-e0d4-44a9-9617-6b93fca92806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:34.694156 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637dcdf6-e0d4-44a9-9617-6b93fca92806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:34.694156 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637dcdf6-e0d4-44a9-9617-6b93fca92806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:34.694156 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637dcdf6-e0d4-44a9-9617-6b93fca92806 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:34.718659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1178/2365] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:52:34.726600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891ab01-121c-4336-8d3a-a0779b4b0c72 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:34.738425 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891ab01-121c-4336-8d3a-a0779b4b0c72 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:34.743978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934b14a-cbfd-4e11-82ff-33e640e4f0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:34.744680 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934b14a-cbfd-4e11-82ff-33e640e4f0a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:34.745048 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934b14a-cbfd-4e11-82ff-33e640e4f0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:34.745258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934b14a-cbfd-4e11-82ff-33e640e4f0a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:34.747058 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934b14a-cbfd-4e11-82ff-33e640e4f0a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:34.749451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934b14a-cbfd-4e11-82ff-33e640e4f0a2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:34.765738 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1188/2366] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:52:34.766038 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637dcdf6-e0d4-44a9-9617-6b93fca92806 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:34.777715 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637dcdf6-e0d4-44a9-9617-6b93fca92806 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:34.783871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f8c4e-787c-48ee-bdd4-9e850f122526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:34.784775 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f8c4e-787c-48ee-bdd4-9e850f122526 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:34.785061 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f8c4e-787c-48ee-bdd4-9e850f122526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:34.786263 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f8c4e-787c-48ee-bdd4-9e850f122526 admin admin] PATH_INFO: `/v3/projects/5871e25b87254084acc8b74d0a99a5c1/users/4f64ade7e8b3473080037dc73294f4e6/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:34.805842 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f8c4e-787c-48ee-bdd4-9e850f122526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5871e25b87254084acc8b74d0a99a5c1, user_id=4f64ade7e8b3473080037dc73294f4e6, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:34.811992 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f8c4e-787c-48ee-bdd4-9e850f122526 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:34.917752 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1179/2367] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:52:34 2022] PUT /identity/v3/projects/5871e25b87254084acc8b74d0a99a5c1/users/4f64ade7e8b3473080037dc73294f4e6/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:34.928114 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb826435-66f9-43e4-8578-f887ed991a85 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:34.928404 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb826435-66f9-43e4-8578-f887ed991a85 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:34.928684 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb826435-66f9-43e4-8578-f887ed991a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:34.960199 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-eb826435-66f9-43e4-8578-f887ed991a85 None None] MFA Rules not processed for user `4f64ade7e8b3473080037dc73294f4e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:34.961981 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-eb826435-66f9-43e4-8578-f887ed991a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:35.089712 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1189/2368] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:52:35.094552 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f8c4e-787c-48ee-bdd4-9e850f122526 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:35.195928 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f8c4e-787c-48ee-bdd4-9e850f122526 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:35.201617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbcb198-2153-4e3b-a67e-176a888caab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:35.203381 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcb198-2153-4e3b-a67e-176a888caab6 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:35.209915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcb198-2153-4e3b-a67e-176a888caab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:35.209915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcb198-2153-4e3b-a67e-176a888caab6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:35.209915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbcb198-2153-4e3b-a67e-176a888caab6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:35.209915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbcb198-2153-4e3b-a67e-176a888caab6 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:35.307414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1180/2369] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:52:35 2022] POST /identity/v3/projects => generated 365 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:52:35.312571 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb826435-66f9-43e4-8578-f887ed991a85 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:35.326002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb826435-66f9-43e4-8578-f887ed991a85 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:35.392160 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:35.410602 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d227c5-c1bb-4243-a281-c08424e16c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:35.412253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d227c5-c1bb-4243-a281-c08424e16c93 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:35.412753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d227c5-c1bb-4243-a281-c08424e16c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:35.413261 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d227c5-c1bb-4243-a281-c08424e16c93 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:35.414622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d227c5-c1bb-4243-a281-c08424e16c93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:35.418824 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d227c5-c1bb-4243-a281-c08424e16c93 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:35.538659 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1190/2370] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:52:35 2022] POST /identity/v3/users => generated 289 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:52:35.543966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbcb198-2153-4e3b-a67e-176a888caab6 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:35.565980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbcb198-2153-4e3b-a67e-176a888caab6 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:35.582673 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d903037-2f04-41ac-bc91-4452c13f3739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:35.584655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d903037-2f04-41ac-bc91-4452c13f3739 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:35.585998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d903037-2f04-41ac-bc91-4452c13f3739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:35.586745 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d903037-2f04-41ac-bc91-4452c13f3739 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:35.588947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d903037-2f04-41ac-bc91-4452c13f3739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:35.592383 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d903037-2f04-41ac-bc91-4452c13f3739 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:35.639237 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1181/2371] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:52:35.645617 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d227c5-c1bb-4243-a281-c08424e16c93 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:35.669247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d227c5-c1bb-4243-a281-c08424e16c93 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:35.677971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380f302-106b-454d-b12c-e8b712cbb6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:35.678597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380f302-106b-454d-b12c-e8b712cbb6af admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:35.678843 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380f302-106b-454d-b12c-e8b712cbb6af admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:35.679149 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380f302-106b-454d-b12c-e8b712cbb6af admin admin] PATH_INFO: `/v3/projects/8d4958590bbd41288a9295021f4683e3/users/56ab4f08d153489e8f7ed43fe55f2667/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:35.702633 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380f302-106b-454d-b12c-e8b712cbb6af admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d4958590bbd41288a9295021f4683e3, user_id=56ab4f08d153489e8f7ed43fe55f2667, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:35.709692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380f302-106b-454d-b12c-e8b712cbb6af admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:35.816253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1191/2372] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:52:35 2022] PUT /identity/v3/projects/8d4958590bbd41288a9295021f4683e3/users/56ab4f08d153489e8f7ed43fe55f2667/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:52:35.818982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d903037-2f04-41ac-bc91-4452c13f3739 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:35.844206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d903037-2f04-41ac-bc91-4452c13f3739 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:35.872338 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:35.967117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d16d9-ce3b-4c07-8b0f-bf30d6f8b25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:35.968583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d16d9-ce3b-4c07-8b0f-bf30d6f8b25f admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:35.969952 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d16d9-ce3b-4c07-8b0f-bf30d6f8b25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:35.970297 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d16d9-ce3b-4c07-8b0f-bf30d6f8b25f admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:35.971008 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d16d9-ce3b-4c07-8b0f-bf30d6f8b25f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:35.973771 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d16d9-ce3b-4c07-8b0f-bf30d6f8b25f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:35.992865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1182/2373] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:52:35.994493 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380f302-106b-454d-b12c-e8b712cbb6af admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:36.018898 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327d3c1-a685-498b-b195-1ad9bf9dffa0 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:36.019396 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327d3c1-a685-498b-b195-1ad9bf9dffa0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:36.020345 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327d3c1-a685-498b-b195-1ad9bf9dffa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:36.021620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380f302-106b-454d-b12c-e8b712cbb6af admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:36.035376 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b9da8-aeaf-48cf-a320-295620b6ceed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:36.036356 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b9da8-aeaf-48cf-a320-295620b6ceed admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:36.036675 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b9da8-aeaf-48cf-a320-295620b6ceed admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:36.036934 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b9da8-aeaf-48cf-a320-295620b6ceed admin admin] PATH_INFO: `/v3/projects/8d4958590bbd41288a9295021f4683e3/users/56ab4f08d153489e8f7ed43fe55f2667/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:36.039406 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b9da8-aeaf-48cf-a320-295620b6ceed admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d4958590bbd41288a9295021f4683e3, user_id=56ab4f08d153489e8f7ed43fe55f2667, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:36.044928 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b9da8-aeaf-48cf-a320-295620b6ceed admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:36.059506 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1192/2374] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:52:35 2022] PUT /identity/v3/projects/8d4958590bbd41288a9295021f4683e3/users/56ab4f08d153489e8f7ed43fe55f2667/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:36.077677 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3785baf-84ac-4921-9b0c-db802f2944ed None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:36.077954 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3785baf-84ac-4921-9b0c-db802f2944ed None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:36.079766 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3785baf-84ac-4921-9b0c-db802f2944ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:36.097621 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-7327d3c1-a685-498b-b195-1ad9bf9dffa0 None None] MFA Rules not processed for user `2b03f8b8c51e472bbd4cf0afc9c6b632`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:36.114856 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7327d3c1-a685-498b-b195-1ad9bf9dffa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:36.154837 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f3785baf-84ac-4921-9b0c-db802f2944ed None None] MFA Rules not processed for user `56ab4f08d153489e8f7ed43fe55f2667`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:36.156380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f3785baf-84ac-4921-9b0c-db802f2944ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:36.259871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1183/2375] 10.222.0.57 () {58 vars in 967 bytes} [Mon Jul 18 08:52:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:52:36.278038 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327d3c1-a685-498b-b195-1ad9bf9dffa0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:36.280038 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7327d3c1-a685-498b-b195-1ad9bf9dffa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:36.307259 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1193/2376] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:36 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:36.322879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3785baf-84ac-4921-9b0c-db802f2944ed None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:36.325639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f3785baf-84ac-4921-9b0c-db802f2944ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:36.422096 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3785baf-84ac-4921-9b0c-db802f2944ed None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:36.428027 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06446c23-5eca-4043-abc6-94c21fe07f09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:36.428291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327d3c1-a685-498b-b195-1ad9bf9dffa0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:36.428755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06446c23-5eca-4043-abc6-94c21fe07f09 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:36.429027 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06446c23-5eca-4043-abc6-94c21fe07f09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:36.429312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06446c23-5eca-4043-abc6-94c21fe07f09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:36.435150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f0e84-14be-4d2e-bc00-30ebc58cc9b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09a5c2e804504038817cad76544dce5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:36.436124 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f0e84-14be-4d2e-bc00-30ebc58cc9b2 service swift] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:36.436419 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f0e84-14be-4d2e-bc00-30ebc58cc9b2 service swift] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:36.436712 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f0e84-14be-4d2e-bc00-30ebc58cc9b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:36.450842 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f0e84-14be-4d2e-bc00-30ebc58cc9b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:36.452917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f0e84-14be-4d2e-bc00-30ebc58cc9b2 service swift] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:36.461114 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06446c23-5eca-4043-abc6-94c21fe07f09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:36.463083 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06446c23-5eca-4043-abc6-94c21fe07f09 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:36.473522 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1184/2377] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Jul 18 08:52:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 08:52:36.489181 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1194/2378] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:52:36 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:52:36.895625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f0e84-14be-4d2e-bc00-30ebc58cc9b2 service swift] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:36.905858 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f0e84-14be-4d2e-bc00-30ebc58cc9b2 service swift] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:36.910177 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715ed29-09e8-4bbb-923c-8b0cfc309ef0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:36.910837 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715ed29-09e8-4bbb-923c-8b0cfc309ef0 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:36.911123 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715ed29-09e8-4bbb-923c-8b0cfc309ef0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:36.911400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715ed29-09e8-4bbb-923c-8b0cfc309ef0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:36.960474 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:36.989471 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715ed29-09e8-4bbb-923c-8b0cfc309ef0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:36.990510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715ed29-09e8-4bbb-923c-8b0cfc309ef0 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:37.010172 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1185/2379] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:52:36 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:52:39.701231 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c76c9-2492-48c2-afaf-330b52d4a1aa None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:39.702020 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c76c9-2492-48c2-afaf-330b52d4a1aa None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:39.702020 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c76c9-2492-48c2-afaf-330b52d4a1aa None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:39.703508 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1195/2380] 10.222.0.57 () {54 vars in 921 bytes} [Mon Jul 18 08:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:52:39.712861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d8f55-19f7-41e6-89e7-9b0c0b59c284 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:39.713322 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d8f55-19f7-41e6-89e7-9b0c0b59c284 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:39.713558 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d8f55-19f7-41e6-89e7-9b0c0b59c284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:39.767582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-079d8f55-19f7-41e6-89e7-9b0c0b59c284 None None] MFA Rules not processed for user `84cba474fc7c470ea119de697b9d7ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:39.768863 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-079d8f55-19f7-41e6-89e7-9b0c0b59c284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:39.911338 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1186/2381] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Jul 18 08:52:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:39.924179 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c76c9-2492-48c2-afaf-330b52d4a1aa None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:39.927102 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ee6c76c9-2492-48c2-afaf-330b52d4a1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:39.974579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:40.010813 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c76c9-2492-48c2-afaf-330b52d4a1aa None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:40.014697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a238bc-9c60-4f8c-83d8-c0108f4c2107 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '694088ed30b94bad9d3b16f3766252df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:40.015352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a238bc-9c60-4f8c-83d8-c0108f4c2107 service glance] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:40.015635 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a238bc-9c60-4f8c-83d8-c0108f4c2107 service glance] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:40.015942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a238bc-9c60-4f8c-83d8-c0108f4c2107 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:40.027876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a238bc-9c60-4f8c-83d8-c0108f4c2107 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:40.029509 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a238bc-9c60-4f8c-83d8-c0108f4c2107 service glance] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:40.041751 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1196/2382] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:52:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:52:40.303405 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b207d4-4c99-4132-8537-bdbb2aeb2626 None None] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:40.303881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b207d4-4c99-4132-8537-bdbb2aeb2626 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:40.304292 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b207d4-4c99-4132-8537-bdbb2aeb2626 None None] PATH_INFO: `/` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:40.305816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1187/2383] 10.222.0.57 () {54 vars in 921 bytes} [Mon Jul 18 08:52:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 08:52:40.311374 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586a7ba-075c-4cfe-a941-479a030f2f35 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:40.311900 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586a7ba-075c-4cfe-a941-479a030f2f35 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:40.312180 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586a7ba-075c-4cfe-a941-479a030f2f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:40.334830 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-3586a7ba-075c-4cfe-a941-479a030f2f35 None None] MFA Rules not processed for user `24a6b794f5774e35b67e9ec4358ffc66`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:40.336120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3586a7ba-075c-4cfe-a941-479a030f2f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:40.438251 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1197/2384] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Jul 18 08:52:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:40.449269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b207d4-4c99-4132-8537-bdbb2aeb2626 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:40.451207 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-90b207d4-4c99-4132-8537-bdbb2aeb2626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:40.474992 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b207d4-4c99-4132-8537-bdbb2aeb2626 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:40.481520 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc26e40-5a76-4e2f-a395-e3640b42cdf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:40.482562 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc26e40-5a76-4e2f-a395-e3640b42cdf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:40.483063 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc26e40-5a76-4e2f-a395-e3640b42cdf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:40.483539 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc26e40-5a76-4e2f-a395-e3640b42cdf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:40.503398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc26e40-5a76-4e2f-a395-e3640b42cdf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:40.505275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc26e40-5a76-4e2f-a395-e3640b42cdf1 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:40.516244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1188/2385] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:52:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:52:40.634199 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425933f1-0112-422b-b292-fc23020913d5 None None] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:40.634492 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425933f1-0112-422b-b292-fc23020913d5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:40.634773 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425933f1-0112-422b-b292-fc23020913d5 None None] PATH_INFO: `/` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:40.636322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1198/2386] 10.222.0.57 () {54 vars in 921 bytes} [Mon Jul 18 08:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 08:52:40.645409 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a62b9-7693-4e6e-b6a1-7f8f342d6ed3 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:40.645653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a62b9-7693-4e6e-b6a1-7f8f342d6ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:40.645763 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a62b9-7693-4e6e-b6a1-7f8f342d6ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:40.666521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-301a62b9-7693-4e6e-b6a1-7f8f342d6ed3 None None] MFA Rules not processed for user `84cba474fc7c470ea119de697b9d7ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:40.667876 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-301a62b9-7693-4e6e-b6a1-7f8f342d6ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:40.678488 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1189/2387] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Jul 18 08:52:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:40.690362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425933f1-0112-422b-b292-fc23020913d5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:40.705238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425933f1-0112-422b-b292-fc23020913d5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:40.713015 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4fa85-f0a3-4b8f-98c1-b413aa644d49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:40.713257 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4fa85-f0a3-4b8f-98c1-b413aa644d49 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:40.713438 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4fa85-f0a3-4b8f-98c1-b413aa644d49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:40.713918 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4fa85-f0a3-4b8f-98c1-b413aa644d49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:40.727009 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4fa85-f0a3-4b8f-98c1-b413aa644d49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:40.729158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4fa85-f0a3-4b8f-98c1-b413aa644d49 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:40.745902 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1199/2388] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:52:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:52:45.972268 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a62b9-7693-4e6e-b6a1-7f8f342d6ed3 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:45.989359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a62b9-7693-4e6e-b6a1-7f8f342d6ed3 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:45.994087 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1aa69-9070-4795-84c0-64e6553fb736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:45.994819 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1aa69-9070-4795-84c0-64e6553fb736 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:45.995133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1aa69-9070-4795-84c0-64e6553fb736 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:45.995411 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1aa69-9070-4795-84c0-64e6553fb736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:45.997560 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-40f1aa69-9070-4795-84c0-64e6553fb736 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:46.086737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1aa69-9070-4795-84c0-64e6553fb736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:46.089223 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1aa69-9070-4795-84c0-64e6553fb736 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:46.110494 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1190/2389] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:52:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:52:46.876453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667abff-9b66-46cd-ae60-a3259707ce0a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:46.876453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667abff-9b66-46cd-ae60-a3259707ce0a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:46.876453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667abff-9b66-46cd-ae60-a3259707ce0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:46.895766 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7667abff-9b66-46cd-ae60-a3259707ce0a None None] MFA Rules not processed for user `2b03f8b8c51e472bbd4cf0afc9c6b632`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:46.897456 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7667abff-9b66-46cd-ae60-a3259707ce0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:46.916221 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1200/2390] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Jul 18 08:52:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:46.922090 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1aa69-9070-4795-84c0-64e6553fb736 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:46.942270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1aa69-9070-4795-84c0-64e6553fb736 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:46.950021 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d9646-83a7-403f-9d63-bb1147de7c76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09a5c2e804504038817cad76544dce5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:46.951979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d9646-83a7-403f-9d63-bb1147de7c76 service swift] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:46.952485 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d9646-83a7-403f-9d63-bb1147de7c76 service swift] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:46.952971 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d9646-83a7-403f-9d63-bb1147de7c76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:46.967895 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d9646-83a7-403f-9d63-bb1147de7c76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:46.970571 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d9646-83a7-403f-9d63-bb1147de7c76 service swift] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:46.979285 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1191/2391] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Jul 18 08:52:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 08:52:49.004070 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7667abff-9b66-46cd-ae60-a3259707ce0a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:49.005583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7667abff-9b66-46cd-ae60-a3259707ce0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:49.155301 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7667abff-9b66-46cd-ae60-a3259707ce0a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:49.165596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd461d-c9cb-441c-9832-bc61dd6eeed8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd606fd7ef269460a8110223932f02d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:49.166406 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd461d-c9cb-441c-9832-bc61dd6eeed8 service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:49.167870 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd461d-c9cb-441c-9832-bc61dd6eeed8 service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:49.168297 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd461d-c9cb-441c-9832-bc61dd6eeed8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:49.170832 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-10fd461d-c9cb-441c-9832-bc61dd6eeed8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:49.199655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d9646-83a7-403f-9d63-bb1147de7c76 service swift] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:49.200219 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd461d-c9cb-441c-9832-bc61dd6eeed8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:49.202531 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c75d9646-83a7-403f-9d63-bb1147de7c76 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:49.210234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd461d-c9cb-441c-9832-bc61dd6eeed8 service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:49.224834 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d9646-83a7-403f-9d63-bb1147de7c76 service swift] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:49.234888 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4902b6-c843-40cc-a086-3120460655a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:49.236120 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4902b6-c843-40cc-a086-3120460655a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:49.236598 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4902b6-c843-40cc-a086-3120460655a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:49.239473 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4902b6-c843-40cc-a086-3120460655a9 admin admin] PATH_INFO: `/v3/users/7e134a318a264a1fb870e69de07b8db8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:49.240278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4902b6-c843-40cc-a086-3120460655a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e134a318a264a1fb870e69de07b8db8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:49.243502 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4902b6-c843-40cc-a086-3120460655a9 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:49.264490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1201/2392] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:52:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:52:49.379910 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:49.489584 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2b4902b6-c843-40cc-a086-3120460655a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e134a318a264a1fb870e69de07b8db8'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:49.492825 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2b4902b6-c843-40cc-a086-3120460655a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e134a318a264a1fb870e69de07b8db8'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:49.499274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2b4902b6-c843-40cc-a086-3120460655a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e134a318a264a1fb870e69de07b8db8'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:49.533359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2b4902b6-c843-40cc-a086-3120460655a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e134a318a264a1fb870e69de07b8db8'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:49.552507 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2b4902b6-c843-40cc-a086-3120460655a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e134a318a264a1fb870e69de07b8db8'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:49.685588 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1192/2393] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:52:49 2022] DELETE /identity/v3/users/7e134a318a264a1fb870e69de07b8db8 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:50.269035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd461d-c9cb-441c-9832-bc61dd6eeed8 service placement] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:50.281332 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd461d-c9cb-441c-9832-bc61dd6eeed8 service placement] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:50.361583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:50.373286 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e01d65-3265-41be-b6a3-f7498d4bdc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:50.375004 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e01d65-3265-41be-b6a3-f7498d4bdc48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:50.376245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e01d65-3265-41be-b6a3-f7498d4bdc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:50.376864 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e01d65-3265-41be-b6a3-f7498d4bdc48 admin admin] PATH_INFO: `/v3/users/4f64ade7e8b3473080037dc73294f4e6` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:50.378452 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e01d65-3265-41be-b6a3-f7498d4bdc48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f64ade7e8b3473080037dc73294f4e6)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:50.380845 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e01d65-3265-41be-b6a3-f7498d4bdc48 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:50.488789 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4902b6-c843-40cc-a086-3120460655a9 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:50.494391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2b4902b6-c843-40cc-a086-3120460655a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:50.511442 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4902b6-c843-40cc-a086-3120460655a9 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:50.530509 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a26ec-9285-4734-a0ae-3ab330577e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:50.535514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a26ec-9285-4734-a0ae-3ab330577e40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:50.536073 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a26ec-9285-4734-a0ae-3ab330577e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:50.536577 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a26ec-9285-4734-a0ae-3ab330577e40 admin admin] PATH_INFO: `/v3/projects/0a75147b684f470890244abb41bc7d67` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:50.539722 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a26ec-9285-4734-a0ae-3ab330577e40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a75147b684f470890244abb41bc7d67)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:50.541844 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a26ec-9285-4734-a0ae-3ab330577e40 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:50.699532 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-53e01d65-3265-41be-b6a3-f7498d4bdc48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f64ade7e8b3473080037dc73294f4e6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:50.700561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-53e01d65-3265-41be-b6a3-f7498d4bdc48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f64ade7e8b3473080037dc73294f4e6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:50.701995 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-53e01d65-3265-41be-b6a3-f7498d4bdc48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f64ade7e8b3473080037dc73294f4e6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:50.748721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-53e01d65-3265-41be-b6a3-f7498d4bdc48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f64ade7e8b3473080037dc73294f4e6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:50.770074 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-53e01d65-3265-41be-b6a3-f7498d4bdc48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f64ade7e8b3473080037dc73294f4e6'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:50.779002 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4b8a26ec-9285-4734-a0ae-3ab330577e40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a75147b684f470890244abb41bc7d67', 'request_id': 'req-4b8a26ec-9285-4734-a0ae-3ab330577e40'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:50.810746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4b8a26ec-9285-4734-a0ae-3ab330577e40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a75147b684f470890244abb41bc7d67', 'request_id': 'req-4b8a26ec-9285-4734-a0ae-3ab330577e40'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:50.861440 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4b8a26ec-9285-4734-a0ae-3ab330577e40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a75147b684f470890244abb41bc7d67', 'request_id': 'req-4b8a26ec-9285-4734-a0ae-3ab330577e40'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:50.908544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1202/2394] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:52:50 2022] DELETE /identity/v3/users/4f64ade7e8b3473080037dc73294f4e6 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:50.911929 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b49d1-8a7c-487a-a536-8d556da4cb31 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:50.912964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b49d1-8a7c-487a-a536-8d556da4cb31 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:50.913420 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b49d1-8a7c-487a-a536-8d556da4cb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:50.935567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-390b49d1-8a7c-487a-a536-8d556da4cb31 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:50.942935 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-390b49d1-8a7c-487a-a536-8d556da4cb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:51.024938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4b8a26ec-9285-4734-a0ae-3ab330577e40 admin admin] The token cache is being invalidate because project 0a75147b684f470890244abb41bc7d67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:52:51.025702 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4b8a26ec-9285-4734-a0ae-3ab330577e40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:51.031223 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4b8a26ec-9285-4734-a0ae-3ab330577e40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:51.034423 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1193/2395] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:52:50 2022] DELETE /identity/v3/projects/0a75147b684f470890244abb41bc7d67 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:52:51.039216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a26ec-9285-4734-a0ae-3ab330577e40 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:51.041872 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4b8a26ec-9285-4734-a0ae-3ab330577e40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:51.062111 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1203/2396] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:51.083062 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b49d1-8a7c-487a-a536-8d556da4cb31 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:51.086636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-390b49d1-8a7c-487a-a536-8d556da4cb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:51.153032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a26ec-9285-4734-a0ae-3ab330577e40 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:51.165724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad919607-b637-40a9-ad66-98becb49b069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:51.166613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad919607-b637-40a9-ad66-98becb49b069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:51.172805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad919607-b637-40a9-ad66-98becb49b069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:51.172805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad919607-b637-40a9-ad66-98becb49b069 admin admin] PATH_INFO: `/v3/users/60cec368ad5b4f9a93fbfdfd92f55913` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:51.172805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad919607-b637-40a9-ad66-98becb49b069 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60cec368ad5b4f9a93fbfdfd92f55913)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:51.173363 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad919607-b637-40a9-ad66-98becb49b069 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:51.204204 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b49d1-8a7c-487a-a536-8d556da4cb31 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:51.208814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e42a81-c39b-44ac-9fcc-0c10a8cee9ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:51.209471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e42a81-c39b-44ac-9fcc-0c10a8cee9ad service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:51.209868 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e42a81-c39b-44ac-9fcc-0c10a8cee9ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:51.210014 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e42a81-c39b-44ac-9fcc-0c10a8cee9ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:51.222606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e42a81-c39b-44ac-9fcc-0c10a8cee9ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:51.225246 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e42a81-c39b-44ac-9fcc-0c10a8cee9ad service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:51.242385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1204/2397] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:52:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:52:51.280582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:51.403525 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ad919607-b637-40a9-ad66-98becb49b069 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60cec368ad5b4f9a93fbfdfd92f55913'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:51.405413 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ad919607-b637-40a9-ad66-98becb49b069 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60cec368ad5b4f9a93fbfdfd92f55913'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:51.408277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ad919607-b637-40a9-ad66-98becb49b069 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60cec368ad5b4f9a93fbfdfd92f55913'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:51.436338 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ad919607-b637-40a9-ad66-98becb49b069 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60cec368ad5b4f9a93fbfdfd92f55913'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:51.447063 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ad919607-b637-40a9-ad66-98becb49b069 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60cec368ad5b4f9a93fbfdfd92f55913'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:51.526996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1194/2398] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:52:51 2022] DELETE /identity/v3/users/60cec368ad5b4f9a93fbfdfd92f55913 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:52:51.787825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e42a81-c39b-44ac-9fcc-0c10a8cee9ad service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:51.790975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-06e42a81-c39b-44ac-9fcc-0c10a8cee9ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:51.821747 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e42a81-c39b-44ac-9fcc-0c10a8cee9ad service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:51.853658 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:51.972976 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519417d1-ea26-4a67-a793-0075e4c11dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:51.974582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519417d1-ea26-4a67-a793-0075e4c11dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:51.974880 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519417d1-ea26-4a67-a793-0075e4c11dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:51.975817 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519417d1-ea26-4a67-a793-0075e4c11dd4 admin admin] PATH_INFO: `/v3/projects/5871e25b87254084acc8b74d0a99a5c1` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:51.977253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519417d1-ea26-4a67-a793-0075e4c11dd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5871e25b87254084acc8b74d0a99a5c1)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:51.979232 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519417d1-ea26-4a67-a793-0075e4c11dd4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:52.037691 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad919607-b637-40a9-ad66-98becb49b069 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:52.039301 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ad919607-b637-40a9-ad66-98becb49b069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:52.060541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad919607-b637-40a9-ad66-98becb49b069 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:52.066747 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74d537-95c8-42a5-85e7-43d7b2d55b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:52.082622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74d537-95c8-42a5-85e7-43d7b2d55b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:52.082999 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74d537-95c8-42a5-85e7-43d7b2d55b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:52.083282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74d537-95c8-42a5-85e7-43d7b2d55b15 admin admin] PATH_INFO: `/v3/projects/f3f54d7971b84445961ce311d6e2eb21` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:52.089272 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74d537-95c8-42a5-85e7-43d7b2d55b15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3f54d7971b84445961ce311d6e2eb21)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:52.091724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74d537-95c8-42a5-85e7-43d7b2d55b15 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:52.159187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-519417d1-ea26-4a67-a793-0075e4c11dd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5871e25b87254084acc8b74d0a99a5c1', 'request_id': 'req-519417d1-ea26-4a67-a793-0075e4c11dd4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:52.178826 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-519417d1-ea26-4a67-a793-0075e4c11dd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5871e25b87254084acc8b74d0a99a5c1', 'request_id': 'req-519417d1-ea26-4a67-a793-0075e4c11dd4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:52.202879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-519417d1-ea26-4a67-a793-0075e4c11dd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5871e25b87254084acc8b74d0a99a5c1', 'request_id': 'req-519417d1-ea26-4a67-a793-0075e4c11dd4'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:52.324828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2d74d537-95c8-42a5-85e7-43d7b2d55b15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3f54d7971b84445961ce311d6e2eb21', 'request_id': 'req-2d74d537-95c8-42a5-85e7-43d7b2d55b15'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:52.346639 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2d74d537-95c8-42a5-85e7-43d7b2d55b15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3f54d7971b84445961ce311d6e2eb21', 'request_id': 'req-2d74d537-95c8-42a5-85e7-43d7b2d55b15'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:52.355399 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2d74d537-95c8-42a5-85e7-43d7b2d55b15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3f54d7971b84445961ce311d6e2eb21', 'request_id': 'req-2d74d537-95c8-42a5-85e7-43d7b2d55b15'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:52.419914 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-519417d1-ea26-4a67-a793-0075e4c11dd4 admin admin] The token cache is being invalidate because project 5871e25b87254084acc8b74d0a99a5c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:52:52.419914 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-519417d1-ea26-4a67-a793-0075e4c11dd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:52.419914 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-519417d1-ea26-4a67-a793-0075e4c11dd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:52.419914 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1205/2399] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:52:51 2022] DELETE /identity/v3/projects/5871e25b87254084acc8b74d0a99a5c1 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:52:52.419914 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519417d1-ea26-4a67-a793-0075e4c11dd4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:52.424911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-519417d1-ea26-4a67-a793-0075e4c11dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:52.486530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2d74d537-95c8-42a5-85e7-43d7b2d55b15 admin admin] The token cache is being invalidate because project f3f54d7971b84445961ce311d6e2eb21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:52:52.486818 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2d74d537-95c8-42a5-85e7-43d7b2d55b15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:52.487491 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2d74d537-95c8-42a5-85e7-43d7b2d55b15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:52.489896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1195/2400] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:52:52 2022] DELETE /identity/v3/projects/f3f54d7971b84445961ce311d6e2eb21 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:52:52.490774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74d537-95c8-42a5-85e7-43d7b2d55b15 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:52.510991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2d74d537-95c8-42a5-85e7-43d7b2d55b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:52.607070 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519417d1-ea26-4a67-a793-0075e4c11dd4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:52.674058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74d537-95c8-42a5-85e7-43d7b2d55b15 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:52.682845 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ea6d7-4353-4d47-b50a-6ba3e312163b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:52.687312 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ea6d7-4353-4d47-b50a-6ba3e312163b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:52.688518 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ea6d7-4353-4d47-b50a-6ba3e312163b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:52.688970 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ea6d7-4353-4d47-b50a-6ba3e312163b admin admin] PATH_INFO: `/v3/users/56ab4f08d153489e8f7ed43fe55f2667` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:52.690682 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ea6d7-4353-4d47-b50a-6ba3e312163b admin admin] RBAC: Authorizing `identity:delete_user(user_id=56ab4f08d153489e8f7ed43fe55f2667)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:52.693864 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ea6d7-4353-4d47-b50a-6ba3e312163b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:52.710313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:52.738125 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81839457-5e3b-4441-a5ba-f93ecee59e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:52.739648 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81839457-5e3b-4441-a5ba-f93ecee59e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:52.740087 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81839457-5e3b-4441-a5ba-f93ecee59e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:52.740520 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81839457-5e3b-4441-a5ba-f93ecee59e1c admin admin] PATH_INFO: `/v3/users/94eed85a730b46dcb1cf694a3ebf2d3a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:52.742132 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81839457-5e3b-4441-a5ba-f93ecee59e1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=94eed85a730b46dcb1cf694a3ebf2d3a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:52.744439 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81839457-5e3b-4441-a5ba-f93ecee59e1c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:52.882418 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:52.896655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9b4ea6d7-4353-4d47-b50a-6ba3e312163b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56ab4f08d153489e8f7ed43fe55f2667'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:52.897456 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9b4ea6d7-4353-4d47-b50a-6ba3e312163b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56ab4f08d153489e8f7ed43fe55f2667'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:52.898186 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9b4ea6d7-4353-4d47-b50a-6ba3e312163b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56ab4f08d153489e8f7ed43fe55f2667'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:52.915192 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9b4ea6d7-4353-4d47-b50a-6ba3e312163b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56ab4f08d153489e8f7ed43fe55f2667'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:52.924473 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9b4ea6d7-4353-4d47-b50a-6ba3e312163b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56ab4f08d153489e8f7ed43fe55f2667'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:52.955294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81839457-5e3b-4441-a5ba-f93ecee59e1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94eed85a730b46dcb1cf694a3ebf2d3a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:52.956222 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81839457-5e3b-4441-a5ba-f93ecee59e1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94eed85a730b46dcb1cf694a3ebf2d3a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:52.956929 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81839457-5e3b-4441-a5ba-f93ecee59e1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94eed85a730b46dcb1cf694a3ebf2d3a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:52.988418 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81839457-5e3b-4441-a5ba-f93ecee59e1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94eed85a730b46dcb1cf694a3ebf2d3a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:52.996337 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-81839457-5e3b-4441-a5ba-f93ecee59e1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94eed85a730b46dcb1cf694a3ebf2d3a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:53.026538 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1196/2401] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:52:52 2022] DELETE /identity/v3/users/56ab4f08d153489e8f7ed43fe55f2667 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:53.106369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1206/2402] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:52:52 2022] DELETE /identity/v3/users/94eed85a730b46dcb1cf694a3ebf2d3a => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:53.483168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ea6d7-4353-4d47-b50a-6ba3e312163b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:53.485405 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9b4ea6d7-4353-4d47-b50a-6ba3e312163b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:53.644336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ea6d7-4353-4d47-b50a-6ba3e312163b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:53.790901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0513924-debb-4fbe-bb2f-b04c80d2ad0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:53.791820 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0513924-debb-4fbe-bb2f-b04c80d2ad0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:53.792107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0513924-debb-4fbe-bb2f-b04c80d2ad0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:53.792373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0513924-debb-4fbe-bb2f-b04c80d2ad0d admin admin] PATH_INFO: `/v3/projects/d7367af701b84358a4e5c5d0db473b44` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:53.793660 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0513924-debb-4fbe-bb2f-b04c80d2ad0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7367af701b84358a4e5c5d0db473b44)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:53.795630 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0513924-debb-4fbe-bb2f-b04c80d2ad0d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:53.803762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81839457-5e3b-4441-a5ba-f93ecee59e1c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:53.805433 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-81839457-5e3b-4441-a5ba-f93ecee59e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:53.836580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81839457-5e3b-4441-a5ba-f93ecee59e1c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:53.845176 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f294ab3-097c-4e5b-b174-7998c338cdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:53.848688 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f294ab3-097c-4e5b-b174-7998c338cdd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:53.849134 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f294ab3-097c-4e5b-b174-7998c338cdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:53.849319 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f294ab3-097c-4e5b-b174-7998c338cdd3 admin admin] PATH_INFO: `/v3/projects/8d4958590bbd41288a9295021f4683e3` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:53.853350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f294ab3-097c-4e5b-b174-7998c338cdd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d4958590bbd41288a9295021f4683e3)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:53.855350 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f294ab3-097c-4e5b-b174-7998c338cdd3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:53.946853 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c0513924-debb-4fbe-bb2f-b04c80d2ad0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7367af701b84358a4e5c5d0db473b44', 'request_id': 'req-c0513924-debb-4fbe-bb2f-b04c80d2ad0d'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:53.953817 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:53.964099 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c0513924-debb-4fbe-bb2f-b04c80d2ad0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7367af701b84358a4e5c5d0db473b44', 'request_id': 'req-c0513924-debb-4fbe-bb2f-b04c80d2ad0d'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:53.980324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c0513924-debb-4fbe-bb2f-b04c80d2ad0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7367af701b84358a4e5c5d0db473b44', 'request_id': 'req-c0513924-debb-4fbe-bb2f-b04c80d2ad0d'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:53.985869 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5f294ab3-097c-4e5b-b174-7998c338cdd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d4958590bbd41288a9295021f4683e3', 'request_id': 'req-5f294ab3-097c-4e5b-b174-7998c338cdd3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:53.996760 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5f294ab3-097c-4e5b-b174-7998c338cdd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d4958590bbd41288a9295021f4683e3', 'request_id': 'req-5f294ab3-097c-4e5b-b174-7998c338cdd3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:54.004480 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5f294ab3-097c-4e5b-b174-7998c338cdd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d4958590bbd41288a9295021f4683e3', 'request_id': 'req-5f294ab3-097c-4e5b-b174-7998c338cdd3'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:54.101475 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c0513924-debb-4fbe-bb2f-b04c80d2ad0d admin admin] The token cache is being invalidate because project d7367af701b84358a4e5c5d0db473b44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:52:54.101787 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c0513924-debb-4fbe-bb2f-b04c80d2ad0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:54.102463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c0513924-debb-4fbe-bb2f-b04c80d2ad0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:54.104575 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1197/2403] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:52:53 2022] DELETE /identity/v3/projects/d7367af701b84358a4e5c5d0db473b44 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:54.109350 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008834a-2696-4d4d-a508-c5e1fd091499 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:54.110195 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008834a-2696-4d4d-a508-c5e1fd091499 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:54.110476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008834a-2696-4d4d-a508-c5e1fd091499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:54.131537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0008834a-2696-4d4d-a508-c5e1fd091499 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:54.133032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0008834a-2696-4d4d-a508-c5e1fd091499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:54.179282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5f294ab3-097c-4e5b-b174-7998c338cdd3 admin admin] The token cache is being invalidate because project 8d4958590bbd41288a9295021f4683e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:52:54.179870 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5f294ab3-097c-4e5b-b174-7998c338cdd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:54.181905 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-5f294ab3-097c-4e5b-b174-7998c338cdd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:54.184408 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1207/2404] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:52:53 2022] DELETE /identity/v3/projects/8d4958590bbd41288a9295021f4683e3 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:54.189970 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084f85b-6690-4865-a77f-3c33ca8feed2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:54.190653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084f85b-6690-4865-a77f-3c33ca8feed2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:54.191150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084f85b-6690-4865-a77f-3c33ca8feed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:54.228005 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1198/2405] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:52:54.236960 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b084f85b-6690-4865-a77f-3c33ca8feed2 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:54.237229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd29671-daf9-494a-8570-14da2319dd6a None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:54.237451 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd29671-daf9-494a-8570-14da2319dd6a None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:54.237733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd29671-daf9-494a-8570-14da2319dd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:54.238230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b084f85b-6690-4865-a77f-3c33ca8feed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:54.251717 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1208/2406] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:52:54.263343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345cea8-6c5e-425c-8a71-146b35b1a1b2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:54.263658 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345cea8-6c5e-425c-8a71-146b35b1a1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:54.263936 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345cea8-6c5e-425c-8a71-146b35b1a1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:54.271371 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6dd29671-daf9-494a-8570-14da2319dd6a None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:54.277443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6dd29671-daf9-494a-8570-14da2319dd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:54.287494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-6345cea8-6c5e-425c-8a71-146b35b1a1b2 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:54.290640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6345cea8-6c5e-425c-8a71-146b35b1a1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:54.296019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1199/2407] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:54.298557 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd29671-daf9-494a-8570-14da2319dd6a None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:54.307340 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1209/2408] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:54.314960 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd29671-daf9-494a-8570-14da2319dd6a None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:54.318138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345cea8-6c5e-425c-8a71-146b35b1a1b2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:54.338318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36920c-85d8-4c44-bd72-14706b49c5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:54.339612 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36920c-85d8-4c44-bd72-14706b49c5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:54.340161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36920c-85d8-4c44-bd72-14706b49c5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:54.340646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36920c-85d8-4c44-bd72-14706b49c5d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:54.342079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36920c-85d8-4c44-bd72-14706b49c5d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:52:54.342226 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345cea8-6c5e-425c-8a71-146b35b1a1b2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:54.342646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36920c-85d8-4c44-bd72-14706b49c5d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:54.346050 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36920c-85d8-4c44-bd72-14706b49c5d8 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:54.348228 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e6c04-f684-4cd4-a271-af995537c510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:54.348921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e6c04-f684-4cd4-a271-af995537c510 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:54.349212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e6c04-f684-4cd4-a271-af995537c510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:54.349498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e6c04-f684-4cd4-a271-af995537c510 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:54.350287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e6c04-f684-4cd4-a271-af995537c510 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:52:54.350821 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e6c04-f684-4cd4-a271-af995537c510 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:54.352472 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e6c04-f684-4cd4-a271-af995537c510 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:54.377921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1210/2409] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:52:54.379075 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e6c04-f684-4cd4-a271-af995537c510 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:54.384018 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1200/2410] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:52:54.390184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36920c-85d8-4c44-bd72-14706b49c5d8 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:54.393433 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e6c04-f684-4cd4-a271-af995537c510 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:54.399131 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd71811-c137-4155-97d0-c9cd16de109e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:54.399868 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd71811-c137-4155-97d0-c9cd16de109e admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:54.400762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd71811-c137-4155-97d0-c9cd16de109e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:54.400762 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd71811-c137-4155-97d0-c9cd16de109e admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:54.401375 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd71811-c137-4155-97d0-c9cd16de109e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:54.403366 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd71811-c137-4155-97d0-c9cd16de109e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:54.405254 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36920c-85d8-4c44-bd72-14706b49c5d8 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:54.424068 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb2d345-27b8-4837-a0e4-a7247a41b5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:54.424893 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2d345-27b8-4837-a0e4-a7247a41b5a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:54.425235 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2d345-27b8-4837-a0e4-a7247a41b5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:54.425541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2d345-27b8-4837-a0e4-a7247a41b5a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:54.426610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb2d345-27b8-4837-a0e4-a7247a41b5a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:54.430389 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb2d345-27b8-4837-a0e4-a7247a41b5a8 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:54.653699 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1211/2411] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:52:54 2022] POST /identity/v3/projects => generated 387 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:52:54.659758 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd71811-c137-4155-97d0-c9cd16de109e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:54.678383 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:54.702374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1201/2412] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:52:54 2022] POST /identity/v3/projects => generated 375 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:52:54.707361 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb2d345-27b8-4837-a0e4-a7247a41b5a8 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:54.767091 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd71811-c137-4155-97d0-c9cd16de109e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:54.865080 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb2d345-27b8-4837-a0e4-a7247a41b5a8 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:54.885185 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d8e26-9b0c-4f26-ad2d-2417848e7def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:54.887214 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8e26-9b0c-4f26-ad2d-2417848e7def admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:54.889075 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8e26-9b0c-4f26-ad2d-2417848e7def admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:54.889476 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8e26-9b0c-4f26-ad2d-2417848e7def admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:54.891767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d8e26-9b0c-4f26-ad2d-2417848e7def admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:54.896189 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d8e26-9b0c-4f26-ad2d-2417848e7def admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:54.943980 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852d762-74a2-4808-9ba3-a0f861a7ce44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:54.944697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852d762-74a2-4808-9ba3-a0f861a7ce44 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:54.944985 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852d762-74a2-4808-9ba3-a0f861a7ce44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:54.945264 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852d762-74a2-4808-9ba3-a0f861a7ce44 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:54.945974 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852d762-74a2-4808-9ba3-a0f861a7ce44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:54.947985 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852d762-74a2-4808-9ba3-a0f861a7ce44 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:55.009129 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1202/2413] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:52:54 2022] POST /identity/v3/users => generated 288 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:52:55.017544 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d8e26-9b0c-4f26-ad2d-2417848e7def admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:55.034971 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d8e26-9b0c-4f26-ad2d-2417848e7def admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:55.041587 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b8533-79d9-4706-8433-ae18608cbaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:55.048301 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b8533-79d9-4706-8433-ae18608cbaa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:55.048301 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b8533-79d9-4706-8433-ae18608cbaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:55.048301 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b8533-79d9-4706-8433-ae18608cbaa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:55.048301 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b8533-79d9-4706-8433-ae18608cbaa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:55.048301 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b8533-79d9-4706-8433-ae18608cbaa2 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:55.060892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1203/2414] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:52:55.066361 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b8533-79d9-4706-8433-ae18608cbaa2 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:55.096567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b8533-79d9-4706-8433-ae18608cbaa2 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:55.110529 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46effa-2e7e-407b-a43d-010f16e71943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:55.112990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46effa-2e7e-407b-a43d-010f16e71943 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:55.113298 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46effa-2e7e-407b-a43d-010f16e71943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:55.114809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46effa-2e7e-407b-a43d-010f16e71943 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:55.115633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46effa-2e7e-407b-a43d-010f16e71943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:55.117344 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46effa-2e7e-407b-a43d-010f16e71943 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:55.148050 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1212/2415] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:52:54 2022] POST /identity/v3/users => generated 294 bytes in 486 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:52:55.150091 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852d762-74a2-4808-9ba3-a0f861a7ce44 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:55.150330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1204/2416] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:52:55.151722 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46effa-2e7e-407b-a43d-010f16e71943 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:55.177685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46effa-2e7e-407b-a43d-010f16e71943 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:55.182769 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852d762-74a2-4808-9ba3-a0f861a7ce44 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:55.183428 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc49987-1153-4599-b776-278fe3a6fc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:55.184304 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc49987-1153-4599-b776-278fe3a6fc9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:55.184611 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc49987-1153-4599-b776-278fe3a6fc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:55.184901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc49987-1153-4599-b776-278fe3a6fc9e admin admin] PATH_INFO: `/v3/projects/a863f588132743b59342ec6931aabf0b/users/fb32461d7cdc4aaf8f1749d03b4ae89e/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:55.191499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d0073-1964-4feb-bd64-ef3b0cc1ef81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:55.192633 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d0073-1964-4feb-bd64-ef3b0cc1ef81 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:55.193115 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d0073-1964-4feb-bd64-ef3b0cc1ef81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:55.195505 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:55.208938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d0073-1964-4feb-bd64-ef3b0cc1ef81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:55.212210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d0073-1964-4feb-bd64-ef3b0cc1ef81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:55.212399 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc49987-1153-4599-b776-278fe3a6fc9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a863f588132743b59342ec6931aabf0b, user_id=fb32461d7cdc4aaf8f1749d03b4ae89e, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:55.214722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d0073-1964-4feb-bd64-ef3b0cc1ef81 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:55.215587 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc49987-1153-4599-b776-278fe3a6fc9e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:55.245126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1213/2417] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:52:55.246514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d0073-1964-4feb-bd64-ef3b0cc1ef81 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:55.271625 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d0073-1964-4feb-bd64-ef3b0cc1ef81 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:55.279416 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c02d6-c7cd-4c9c-a39a-1f73f83062e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:55.280444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c02d6-c7cd-4c9c-a39a-1f73f83062e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:55.280914 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c02d6-c7cd-4c9c-a39a-1f73f83062e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:55.282004 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c02d6-c7cd-4c9c-a39a-1f73f83062e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:55.283183 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c02d6-c7cd-4c9c-a39a-1f73f83062e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:55.285188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c02d6-c7cd-4c9c-a39a-1f73f83062e6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:55.300110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1214/2418] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:52:55.304579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c02d6-c7cd-4c9c-a39a-1f73f83062e6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:55.326116 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c02d6-c7cd-4c9c-a39a-1f73f83062e6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:55.326392 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1205/2419] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:52:55 2022] PUT /identity/v3/projects/a863f588132743b59342ec6931aabf0b/users/fb32461d7cdc4aaf8f1749d03b4ae89e/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:55.330883 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf75ad-3de1-4e1e-9c1b-0722e8daae65 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:55.333628 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf75ad-3de1-4e1e-9c1b-0722e8daae65 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:55.334190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf75ad-3de1-4e1e-9c1b-0722e8daae65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:55.406193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-90bf75ad-3de1-4e1e-9c1b-0722e8daae65 None None] MFA Rules not processed for user `fb32461d7cdc4aaf8f1749d03b4ae89e`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:55.408187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-90bf75ad-3de1-4e1e-9c1b-0722e8daae65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:55.454345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9eafa9-91a5-4f6b-80d5-9a06b7fa69b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:55.455298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9eafa9-91a5-4f6b-80d5-9a06b7fa69b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:55.455591 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9eafa9-91a5-4f6b-80d5-9a06b7fa69b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:55.455923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9eafa9-91a5-4f6b-80d5-9a06b7fa69b6 admin admin] PATH_INFO: `/v3/projects/1e79982e12024706babbc11a018e865d/users/7ed0073dcb64477f89e1e8928258c311/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:55.495018 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9eafa9-91a5-4f6b-80d5-9a06b7fa69b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e79982e12024706babbc11a018e865d, user_id=7ed0073dcb64477f89e1e8928258c311, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:55.498017 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9eafa9-91a5-4f6b-80d5-9a06b7fa69b6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:55.526040 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1206/2420] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:55.541407 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf75ad-3de1-4e1e-9c1b-0722e8daae65 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:55.548035 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-90bf75ad-3de1-4e1e-9c1b-0722e8daae65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:55.611711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1215/2421] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:52:55 2022] PUT /identity/v3/projects/1e79982e12024706babbc11a018e865d/users/7ed0073dcb64477f89e1e8928258c311/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:55.625218 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77957ac3-add7-4953-8450-c5a7a4d11571 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:55.625555 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77957ac3-add7-4953-8450-c5a7a4d11571 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:55.625839 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77957ac3-add7-4953-8450-c5a7a4d11571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:55.649303 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:55.683493 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-77957ac3-add7-4953-8450-c5a7a4d11571 None None] MFA Rules not processed for user `7ed0073dcb64477f89e1e8928258c311`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:55.685778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-77957ac3-add7-4953-8450-c5a7a4d11571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:55.710494 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf75ad-3de1-4e1e-9c1b-0722e8daae65 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:55.828643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1216/2422] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:55.829707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77957ac3-add7-4953-8450-c5a7a4d11571 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:55.851506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a4201-df3c-4e0f-b5db-be75236a3396 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:55.852663 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a4201-df3c-4e0f-b5db-be75236a3396 service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:55.853279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a4201-df3c-4e0f-b5db-be75236a3396 service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:55.854432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a4201-df3c-4e0f-b5db-be75236a3396 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:55.869752 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:55.950735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77957ac3-add7-4953-8450-c5a7a4d11571 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:55.954367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a4201-df3c-4e0f-b5db-be75236a3396 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:55.958048 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a4201-df3c-4e0f-b5db-be75236a3396 service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:55.960232 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b2371-d3c6-4614-972c-0486e2f6b1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:55.961372 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b2371-d3c6-4614-972c-0486e2f6b1fb admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:55.962211 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b2371-d3c6-4614-972c-0486e2f6b1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:55.962785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b2371-d3c6-4614-972c-0486e2f6b1fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:55.964385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b2371-d3c6-4614-972c-0486e2f6b1fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:55.967499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b2371-d3c6-4614-972c-0486e2f6b1fb admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:55.980772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1207/2423] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:52:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:52:56.178507 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a4201-df3c-4e0f-b5db-be75236a3396 service nova] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:56.194184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1217/2424] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:52:55 2022] POST /identity/v3/projects => generated 385 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:52:56.205238 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a4201-df3c-4e0f-b5db-be75236a3396 service nova] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:56.343415 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcb08f-2dab-4f82-9aad-a6ccd7a4be4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:56.344104 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcb08f-2dab-4f82-9aad-a6ccd7a4be4d admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:56.345658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcb08f-2dab-4f82-9aad-a6ccd7a4be4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:56.346135 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcb08f-2dab-4f82-9aad-a6ccd7a4be4d admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:56.347224 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dcb08f-2dab-4f82-9aad-a6ccd7a4be4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:56.349498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dcb08f-2dab-4f82-9aad-a6ccd7a4be4d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:56.452580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1208/2425] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:52:56 2022] POST /identity/v3/users => generated 299 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:52:56.461874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b2371-d3c6-4614-972c-0486e2f6b1fb admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:56.482639 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b2371-d3c6-4614-972c-0486e2f6b1fb admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:56.490144 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79c83d-a097-4058-920b-601c49a9bc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:56.490989 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79c83d-a097-4058-920b-601c49a9bc03 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:56.491378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79c83d-a097-4058-920b-601c49a9bc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:56.491829 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79c83d-a097-4058-920b-601c49a9bc03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:56.506760 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79c83d-a097-4058-920b-601c49a9bc03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:56.508400 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79c83d-a097-4058-920b-601c49a9bc03 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:56.531354 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1218/2426] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:52:56.538104 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcb08f-2dab-4f82-9aad-a6ccd7a4be4d admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:56.576368 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcb08f-2dab-4f82-9aad-a6ccd7a4be4d admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:56.588906 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c1381-b26a-44bc-9d98-260fea965096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:56.591735 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c1381-b26a-44bc-9d98-260fea965096 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:56.592694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c1381-b26a-44bc-9d98-260fea965096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:56.595176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c1381-b26a-44bc-9d98-260fea965096 admin admin] PATH_INFO: `/v3/projects/227ce65aaafa47a5be2d92f5813d0e74/users/da5d4d78f95e4e2881234c21133b3c4e/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:56.642235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79c83d-a097-4058-920b-601c49a9bc03 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:56.660007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c1381-b26a-44bc-9d98-260fea965096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=227ce65aaafa47a5be2d92f5813d0e74, user_id=da5d4d78f95e4e2881234c21133b3c4e, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:56.663518 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c1381-b26a-44bc-9d98-260fea965096 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:56.664106 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79c83d-a097-4058-920b-601c49a9bc03 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:56.688679 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e195a15-51e8-4e42-b188-c141187a6ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:56.688679 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e195a15-51e8-4e42-b188-c141187a6ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:56.688679 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e195a15-51e8-4e42-b188-c141187a6ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:56.688679 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e195a15-51e8-4e42-b188-c141187a6ecd admin admin] PATH_INFO: `/v3/users/fb32461d7cdc4aaf8f1749d03b4ae89e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:56.689090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e195a15-51e8-4e42-b188-c141187a6ecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb32461d7cdc4aaf8f1749d03b4ae89e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:56.689090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e195a15-51e8-4e42-b188-c141187a6ecd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:56.763560 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1209/2427] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:52:56 2022] PUT /identity/v3/projects/227ce65aaafa47a5be2d92f5813d0e74/users/da5d4d78f95e4e2881234c21133b3c4e/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:52:56.768988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c1381-b26a-44bc-9d98-260fea965096 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:56.917299 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c1381-b26a-44bc-9d98-260fea965096 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:56.917299 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38bcd59-1b39-4972-8391-45dbc40ce79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:56.917299 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bcd59-1b39-4972-8391-45dbc40ce79a admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:56.917299 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bcd59-1b39-4972-8391-45dbc40ce79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:56.918772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bcd59-1b39-4972-8391-45dbc40ce79a admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:56.918772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38bcd59-1b39-4972-8391-45dbc40ce79a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:56.918772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38bcd59-1b39-4972-8391-45dbc40ce79a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:56.918772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1210/2428] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:52:56.918772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38bcd59-1b39-4972-8391-45dbc40ce79a admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:56.919445 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38bcd59-1b39-4972-8391-45dbc40ce79a admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:56.936369 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb794a2-a622-4486-b9f2-9ddcf8892770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:56.937412 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb794a2-a622-4486-b9f2-9ddcf8892770 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:56.937574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb794a2-a622-4486-b9f2-9ddcf8892770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:56.938356 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb794a2-a622-4486-b9f2-9ddcf8892770 admin admin] PATH_INFO: `/v3/projects/227ce65aaafa47a5be2d92f5813d0e74/users/da5d4d78f95e4e2881234c21133b3c4e/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:56.940391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb794a2-a622-4486-b9f2-9ddcf8892770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=227ce65aaafa47a5be2d92f5813d0e74, user_id=da5d4d78f95e4e2881234c21133b3c4e, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:56.956876 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb794a2-a622-4486-b9f2-9ddcf8892770 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:56.966030 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:56.986053 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1211/2429] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:52:56 2022] PUT /identity/v3/projects/227ce65aaafa47a5be2d92f5813d0e74/users/da5d4d78f95e4e2881234c21133b3c4e/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:57.010469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106fb1f-b902-4cdd-b8d4-4b8256420b09 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:57.010798 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106fb1f-b902-4cdd-b8d4-4b8256420b09 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:57.011275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106fb1f-b902-4cdd-b8d4-4b8256420b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:57.043035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4e195a15-51e8-4e42-b188-c141187a6ecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb32461d7cdc4aaf8f1749d03b4ae89e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:57.045889 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4e195a15-51e8-4e42-b188-c141187a6ecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb32461d7cdc4aaf8f1749d03b4ae89e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:57.047952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4e195a15-51e8-4e42-b188-c141187a6ecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb32461d7cdc4aaf8f1749d03b4ae89e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:57.078688 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-8106fb1f-b902-4cdd-b8d4-4b8256420b09 None None] MFA Rules not processed for user `da5d4d78f95e4e2881234c21133b3c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:57.079567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4e195a15-51e8-4e42-b188-c141187a6ecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb32461d7cdc4aaf8f1749d03b4ae89e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:57.080678 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8106fb1f-b902-4cdd-b8d4-4b8256420b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:57.082882 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:57.094349 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4e195a15-51e8-4e42-b188-c141187a6ecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb32461d7cdc4aaf8f1749d03b4ae89e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:57.211259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1212/2430] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:52:57.231013 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106fb1f-b902-4cdd-b8d4-4b8256420b09 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:57.233883 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8106fb1f-b902-4cdd-b8d4-4b8256420b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:57.239187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1219/2431] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:52:56 2022] DELETE /identity/v3/users/fb32461d7cdc4aaf8f1749d03b4ae89e => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:57.258124 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08dfbef-6be9-40e2-85fd-b8222d394d04 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:57.258962 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08dfbef-6be9-40e2-85fd-b8222d394d04 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:57.259460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08dfbef-6be9-40e2-85fd-b8222d394d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:57.292013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-c08dfbef-6be9-40e2-85fd-b8222d394d04 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:57.294315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c08dfbef-6be9-40e2-85fd-b8222d394d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:57.392211 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106fb1f-b902-4cdd-b8d4-4b8256420b09 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:57.436502 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1220/2432] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:57.445990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08dfbef-6be9-40e2-85fd-b8222d394d04 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:57.448538 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-c08dfbef-6be9-40e2-85fd-b8222d394d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:57.498420 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a432bf9-6a83-4611-a5a1-6663cf6d28db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:57.502941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a432bf9-6a83-4611-a5a1-6663cf6d28db service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:57.503561 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a432bf9-6a83-4611-a5a1-6663cf6d28db service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:57.504199 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a432bf9-6a83-4611-a5a1-6663cf6d28db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:57.645278 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08dfbef-6be9-40e2-85fd-b8222d394d04 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:57.655689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:57.660195 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa20f0-08ab-492a-8ffc-72833bf01236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:57.660885 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa20f0-08ab-492a-8ffc-72833bf01236 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:57.661214 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa20f0-08ab-492a-8ffc-72833bf01236 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:57.661497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa20f0-08ab-492a-8ffc-72833bf01236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:57.684100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa20f0-08ab-492a-8ffc-72833bf01236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:57.685658 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa20f0-08ab-492a-8ffc-72833bf01236 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:57.689466 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a432bf9-6a83-4611-a5a1-6663cf6d28db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:57.691499 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a432bf9-6a83-4611-a5a1-6663cf6d28db service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:57.717401 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1213/2433] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:52:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:52:57.724540 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1221/2434] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:52:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:52:57.867538 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a432bf9-6a83-4611-a5a1-6663cf6d28db service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:57.870748 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2a432bf9-6a83-4611-a5a1-6663cf6d28db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:57.912744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a432bf9-6a83-4611-a5a1-6663cf6d28db service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:57.923751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ace875-f425-4983-a92f-78384b40798c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:57.924817 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ace875-f425-4983-a92f-78384b40798c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:57.924999 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ace875-f425-4983-a92f-78384b40798c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:57.925205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ace875-f425-4983-a92f-78384b40798c admin admin] PATH_INFO: `/v3/users/7ed0073dcb64477f89e1e8928258c311` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:57.926651 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ace875-f425-4983-a92f-78384b40798c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed0073dcb64477f89e1e8928258c311)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:57.929795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ace875-f425-4983-a92f-78384b40798c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:58.274836 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e6ace875-f425-4983-a92f-78384b40798c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed0073dcb64477f89e1e8928258c311'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:58.275523 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e6ace875-f425-4983-a92f-78384b40798c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed0073dcb64477f89e1e8928258c311'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:58.276370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e6ace875-f425-4983-a92f-78384b40798c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed0073dcb64477f89e1e8928258c311'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:58.305692 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e6ace875-f425-4983-a92f-78384b40798c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed0073dcb64477f89e1e8928258c311'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:58.316489 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-e6ace875-f425-4983-a92f-78384b40798c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed0073dcb64477f89e1e8928258c311'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:58.401233 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1214/2435] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:52:57 2022] DELETE /identity/v3/users/7ed0073dcb64477f89e1e8928258c311 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:52:58.407451 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df361641-e9c2-4ea4-b89b-3a3caef7647a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:58.408841 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df361641-e9c2-4ea4-b89b-3a3caef7647a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:58.409535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df361641-e9c2-4ea4-b89b-3a3caef7647a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:58.434030 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-df361641-e9c2-4ea4-b89b-3a3caef7647a None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:58.435281 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-df361641-e9c2-4ea4-b89b-3a3caef7647a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:58.508316 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1222/2436] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:58.518726 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ace875-f425-4983-a92f-78384b40798c admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:58.520420 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e6ace875-f425-4983-a92f-78384b40798c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:58.545185 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df361641-e9c2-4ea4-b89b-3a3caef7647a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:58.546398 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-df361641-e9c2-4ea4-b89b-3a3caef7647a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:58.565442 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df361641-e9c2-4ea4-b89b-3a3caef7647a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:58.571031 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b63c4-4b68-4e67-8ba9-ee9721bfcf5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:58.571694 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b63c4-4b68-4e67-8ba9-ee9721bfcf5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:58.571984 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b63c4-4b68-4e67-8ba9-ee9721bfcf5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:58.572263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b63c4-4b68-4e67-8ba9-ee9721bfcf5e admin admin] PATH_INFO: `/v3/projects/a863f588132743b59342ec6931aabf0b` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:58.573584 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b63c4-4b68-4e67-8ba9-ee9721bfcf5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a863f588132743b59342ec6931aabf0b)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:58.575742 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b63c4-4b68-4e67-8ba9-ee9721bfcf5e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:58.617367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ace875-f425-4983-a92f-78384b40798c admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:58.620783 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:58.621853 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2c62f-3e38-45ac-a73c-2399862bbd49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:58.623343 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2c62f-3e38-45ac-a73c-2399862bbd49 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:58.623627 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2c62f-3e38-45ac-a73c-2399862bbd49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:58.623956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2c62f-3e38-45ac-a73c-2399862bbd49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:58.635324 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2c62f-3e38-45ac-a73c-2399862bbd49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:58.636879 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2c62f-3e38-45ac-a73c-2399862bbd49 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:58.648404 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1215/2437] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:52:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:52:58.727797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ea8b63c4-4b68-4e67-8ba9-ee9721bfcf5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a863f588132743b59342ec6931aabf0b', 'request_id': 'req-ea8b63c4-4b68-4e67-8ba9-ee9721bfcf5e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:58.734473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ea8b63c4-4b68-4e67-8ba9-ee9721bfcf5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a863f588132743b59342ec6931aabf0b', 'request_id': 'req-ea8b63c4-4b68-4e67-8ba9-ee9721bfcf5e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:58.741922 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ea8b63c4-4b68-4e67-8ba9-ee9721bfcf5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a863f588132743b59342ec6931aabf0b', 'request_id': 'req-ea8b63c4-4b68-4e67-8ba9-ee9721bfcf5e'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:58.853714 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ea8b63c4-4b68-4e67-8ba9-ee9721bfcf5e admin admin] The token cache is being invalidate because project a863f588132743b59342ec6931aabf0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:52:58.854463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ea8b63c4-4b68-4e67-8ba9-ee9721bfcf5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:58.856579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ea8b63c4-4b68-4e67-8ba9-ee9721bfcf5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:58.864016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1223/2438] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:52:58 2022] DELETE /identity/v3/projects/a863f588132743b59342ec6931aabf0b => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:58.865442 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b619981-b035-463b-9dd7-9a349bf82428 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:58.865759 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b619981-b035-463b-9dd7-9a349bf82428 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:58.866012 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b619981-b035-463b-9dd7-9a349bf82428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:58.887723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6b619981-b035-463b-9dd7-9a349bf82428 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:58.888965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6b619981-b035-463b-9dd7-9a349bf82428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:58.973307 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1216/2439] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:52:58.988078 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e29725-a071-4582-9374-f59a7c8b8795 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:58.988883 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e29725-a071-4582-9374-f59a7c8b8795 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:58.989403 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e29725-a071-4582-9374-f59a7c8b8795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:59.022983 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-35e29725-a071-4582-9374-f59a7c8b8795 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:59.025419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-35e29725-a071-4582-9374-f59a7c8b8795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:59.041060 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1224/2440] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:52:59.046095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b619981-b035-463b-9dd7-9a349bf82428 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:59.064765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b619981-b035-463b-9dd7-9a349bf82428 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:59.071375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266261a-d7cc-4263-9dc3-501ab37b34a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:59.072089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266261a-d7cc-4263-9dc3-501ab37b34a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:59.072374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266261a-d7cc-4263-9dc3-501ab37b34a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:59.072649 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266261a-d7cc-4263-9dc3-501ab37b34a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:59.073390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266261a-d7cc-4263-9dc3-501ab37b34a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:52:59.073751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266261a-d7cc-4263-9dc3-501ab37b34a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:59.075343 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266261a-d7cc-4263-9dc3-501ab37b34a3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:59.077553 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:59.105153 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1217/2441] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:52:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:52:59.106530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e29725-a071-4582-9374-f59a7c8b8795 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:59.120413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e29725-a071-4582-9374-f59a7c8b8795 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:59.129698 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f03cc-c3f5-48ea-8b80-871050207a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:59.129698 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f03cc-c3f5-48ea-8b80-871050207a21 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:59.129698 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f03cc-c3f5-48ea-8b80-871050207a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:59.129698 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f03cc-c3f5-48ea-8b80-871050207a21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:59.130095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f03cc-c3f5-48ea-8b80-871050207a21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:59.130095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f03cc-c3f5-48ea-8b80-871050207a21 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:59.223379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266261a-d7cc-4263-9dc3-501ab37b34a3 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:59.229236 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c266261a-d7cc-4263-9dc3-501ab37b34a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:59.252431 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266261a-d7cc-4263-9dc3-501ab37b34a3 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:59.259192 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b646879-9eaf-4850-9d3f-d976aa8163a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:59.260504 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b646879-9eaf-4850-9d3f-d976aa8163a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:59.260992 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b646879-9eaf-4850-9d3f-d976aa8163a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:59.261531 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b646879-9eaf-4850-9d3f-d976aa8163a2 admin admin] PATH_INFO: `/v3/projects/1e79982e12024706babbc11a018e865d` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:59.263158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b646879-9eaf-4850-9d3f-d976aa8163a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e79982e12024706babbc11a018e865d)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:59.266764 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b646879-9eaf-4850-9d3f-d976aa8163a2 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:59.294745 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1225/2442] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:52:59 2022] POST /identity/v3/projects => generated 367 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:52:59.299987 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f03cc-c3f5-48ea-8b80-871050207a21 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:59.385255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f03cc-c3f5-48ea-8b80-871050207a21 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:59.394510 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1166934-6bf0-416a-adc2-7db15eba8133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:59.394510 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1166934-6bf0-416a-adc2-7db15eba8133 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:59.394510 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1166934-6bf0-416a-adc2-7db15eba8133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:59.394510 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1166934-6bf0-416a-adc2-7db15eba8133 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:59.395094 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1166934-6bf0-416a-adc2-7db15eba8133 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:59.395094 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1166934-6bf0-416a-adc2-7db15eba8133 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:59.407889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3b646879-9eaf-4850-9d3f-d976aa8163a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e79982e12024706babbc11a018e865d', 'request_id': 'req-3b646879-9eaf-4850-9d3f-d976aa8163a2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:59.415842 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3b646879-9eaf-4850-9d3f-d976aa8163a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e79982e12024706babbc11a018e865d', 'request_id': 'req-3b646879-9eaf-4850-9d3f-d976aa8163a2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:59.430272 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3b646879-9eaf-4850-9d3f-d976aa8163a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e79982e12024706babbc11a018e865d', 'request_id': 'req-3b646879-9eaf-4850-9d3f-d976aa8163a2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:59.473973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1226/2443] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:52:59 2022] POST /identity/v3/users => generated 284 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:52:59.478078 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1166934-6bf0-416a-adc2-7db15eba8133 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:59.489692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1166934-6bf0-416a-adc2-7db15eba8133 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:59.503068 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3b646879-9eaf-4850-9d3f-d976aa8163a2 admin admin] The token cache is being invalidate because project 1e79982e12024706babbc11a018e865d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:52:59.504158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3b646879-9eaf-4850-9d3f-d976aa8163a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:59.504158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3b646879-9eaf-4850-9d3f-d976aa8163a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:59.507705 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1218/2444] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:52:59 2022] DELETE /identity/v3/projects/1e79982e12024706babbc11a018e865d => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:52:59.510020 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b646879-9eaf-4850-9d3f-d976aa8163a2 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:59.511808 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3b646879-9eaf-4850-9d3f-d976aa8163a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:59.568593 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c371b95-7515-447d-b6e4-0c03eb0f3142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:59.569491 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c371b95-7515-447d-b6e4-0c03eb0f3142 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:59.569875 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c371b95-7515-447d-b6e4-0c03eb0f3142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:59.570295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c371b95-7515-447d-b6e4-0c03eb0f3142 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:59.571277 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c371b95-7515-447d-b6e4-0c03eb0f3142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:59.573078 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c371b95-7515-447d-b6e4-0c03eb0f3142 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:59.589513 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1227/2445] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:52:59.591120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c371b95-7515-447d-b6e4-0c03eb0f3142 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:59.593040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6c371b95-7515-447d-b6e4-0c03eb0f3142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:59.595371 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b646879-9eaf-4850-9d3f-d976aa8163a2 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:59.600460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7120f-0ef1-43dc-8a69-f2b5d5d10375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:59.601717 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7120f-0ef1-43dc-8a69-f2b5d5d10375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:59.602023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7120f-0ef1-43dc-8a69-f2b5d5d10375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:59.602380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7120f-0ef1-43dc-8a69-f2b5d5d10375 admin admin] PATH_INFO: `/v3/users/da5d4d78f95e4e2881234c21133b3c4e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:59.605167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa7120f-0ef1-43dc-8a69-f2b5d5d10375 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da5d4d78f95e4e2881234c21133b3c4e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:59.607428 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa7120f-0ef1-43dc-8a69-f2b5d5d10375 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:59.608209 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c371b95-7515-447d-b6e4-0c03eb0f3142 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:59.614009 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894d2f5-591a-456f-8080-f65fa12b19cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:59.614703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894d2f5-591a-456f-8080-f65fa12b19cb admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:59.614986 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894d2f5-591a-456f-8080-f65fa12b19cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:59.615264 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894d2f5-591a-456f-8080-f65fa12b19cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:59.616811 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894d2f5-591a-456f-8080-f65fa12b19cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:59.617698 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894d2f5-591a-456f-8080-f65fa12b19cb admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:59.619510 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:52:59.631734 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1228/2446] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:52:59.635797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894d2f5-591a-456f-8080-f65fa12b19cb admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:59.651800 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894d2f5-591a-456f-8080-f65fa12b19cb admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:52:59.658514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b80913-160f-4a4d-b041-f8410350ca12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:52:59.659662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b80913-160f-4a4d-b041-f8410350ca12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:59.660088 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b80913-160f-4a4d-b041-f8410350ca12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:59.660473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b80913-160f-4a4d-b041-f8410350ca12 admin admin] PATH_INFO: `/v3/projects/e4dfc36cfd8c4019970a277cb3935d38/users/7780be00665642f291b24f28342f7d82/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:59.682749 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b80913-160f-4a4d-b041-f8410350ca12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4dfc36cfd8c4019970a277cb3935d38, user_id=7780be00665642f291b24f28342f7d82, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:52:59.685744 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b80913-160f-4a4d-b041-f8410350ca12 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:52:59.774159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1229/2447] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:52:59 2022] PUT /identity/v3/projects/e4dfc36cfd8c4019970a277cb3935d38/users/7780be00665642f291b24f28342f7d82/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:52:59.787426 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed79187-3890-4881-816d-473a888b49f6 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:52:59.788013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed79187-3890-4881-816d-473a888b49f6 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:52:59.788162 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed79187-3890-4881-816d-473a888b49f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:52:59.821970 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-aed79187-3890-4881-816d-473a888b49f6 None None] MFA Rules not processed for user `7780be00665642f291b24f28342f7d82`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:52:59.823494 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-aed79187-3890-4881-816d-473a888b49f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:52:59.911876 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2fa7120f-0ef1-43dc-8a69-f2b5d5d10375 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da5d4d78f95e4e2881234c21133b3c4e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:59.915738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2fa7120f-0ef1-43dc-8a69-f2b5d5d10375 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da5d4d78f95e4e2881234c21133b3c4e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:59.915738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2fa7120f-0ef1-43dc-8a69-f2b5d5d10375 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da5d4d78f95e4e2881234c21133b3c4e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:59.933174 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1230/2448] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:52:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:52:59.934203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed79187-3890-4881-816d-473a888b49f6 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:52:59.950861 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2fa7120f-0ef1-43dc-8a69-f2b5d5d10375 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da5d4d78f95e4e2881234c21133b3c4e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:59.963208 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2fa7120f-0ef1-43dc-8a69-f2b5d5d10375 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da5d4d78f95e4e2881234c21133b3c4e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:52:59.971432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:00.056937 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed79187-3890-4881-816d-473a888b49f6 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:00.073752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c6e35-78dc-41c5-a6bf-7096fb143d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:00.076129 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c6e35-78dc-41c5-a6bf-7096fb143d1e admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:00.077398 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c6e35-78dc-41c5-a6bf-7096fb143d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:00.078247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c6e35-78dc-41c5-a6bf-7096fb143d1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:00.080049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c6e35-78dc-41c5-a6bf-7096fb143d1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:00.085441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c6e35-78dc-41c5-a6bf-7096fb143d1e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:00.088937 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1219/2449] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:52:59 2022] DELETE /identity/v3/users/da5d4d78f95e4e2881234c21133b3c4e => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:00.262808 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7120f-0ef1-43dc-8a69-f2b5d5d10375 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:00.264570 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2fa7120f-0ef1-43dc-8a69-f2b5d5d10375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:00.273246 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1231/2450] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:52:59 2022] POST /identity/v3/projects => generated 369 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:00.381310 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7120f-0ef1-43dc-8a69-f2b5d5d10375 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:00.486339 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec54a7-048b-4617-a7bb-6098ef2941c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:00.487138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec54a7-048b-4617-a7bb-6098ef2941c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:00.487558 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec54a7-048b-4617-a7bb-6098ef2941c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:00.487892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec54a7-048b-4617-a7bb-6098ef2941c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:00.489641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec54a7-048b-4617-a7bb-6098ef2941c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:00.492384 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec54a7-048b-4617-a7bb-6098ef2941c2 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:00.628140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1220/2451] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:53:00 2022] POST /identity/v3/users => generated 291 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:00.632666 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c6e35-78dc-41c5-a6bf-7096fb143d1e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:00.643641 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c6e35-78dc-41c5-a6bf-7096fb143d1e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:00.648770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4cf745-34e2-48ce-9be6-420ce292687a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:00.649459 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cf745-34e2-48ce-9be6-420ce292687a admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:00.653391 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cf745-34e2-48ce-9be6-420ce292687a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:00.653391 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cf745-34e2-48ce-9be6-420ce292687a admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:00.653391 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4cf745-34e2-48ce-9be6-420ce292687a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:00.653391 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4cf745-34e2-48ce-9be6-420ce292687a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:00.666804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1232/2452] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:00.671356 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec54a7-048b-4617-a7bb-6098ef2941c2 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:00.683340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec54a7-048b-4617-a7bb-6098ef2941c2 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:00.688580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3709a3-deba-4e23-9cf2-0d3c40d1f221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:00.689489 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3709a3-deba-4e23-9cf2-0d3c40d1f221 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:00.689932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3709a3-deba-4e23-9cf2-0d3c40d1f221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:00.690219 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3709a3-deba-4e23-9cf2-0d3c40d1f221 admin admin] PATH_INFO: `/v3/projects/26b8b90155644c5f8382dc423eae8bf0/users/61564210ef43453794fe6333e2b661d1/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:00.707287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3709a3-deba-4e23-9cf2-0d3c40d1f221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26b8b90155644c5f8382dc423eae8bf0, user_id=61564210ef43453794fe6333e2b661d1, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:00.710725 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3709a3-deba-4e23-9cf2-0d3c40d1f221 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:00.824318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1221/2453] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:53:00 2022] PUT /identity/v3/projects/26b8b90155644c5f8382dc423eae8bf0/users/61564210ef43453794fe6333e2b661d1/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:00.829017 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4cf745-34e2-48ce-9be6-420ce292687a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:00.840288 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4cf745-34e2-48ce-9be6-420ce292687a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:00.859015 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3709a3-deba-4e23-9cf2-0d3c40d1f221 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:00.864953 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-7b3709a3-deba-4e23-9cf2-0d3c40d1f221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:00.918167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:00.969739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1e6ac-1af4-45df-8c7e-48a0a6bde8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:00.970928 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1e6ac-1af4-45df-8c7e-48a0a6bde8e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:00.971394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1e6ac-1af4-45df-8c7e-48a0a6bde8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:00.972086 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1e6ac-1af4-45df-8c7e-48a0a6bde8e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:00.973600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1e6ac-1af4-45df-8c7e-48a0a6bde8e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:00.975647 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1e6ac-1af4-45df-8c7e-48a0a6bde8e9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:00.987197 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3709a3-deba-4e23-9cf2-0d3c40d1f221 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:00.990403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11525280-cdb0-46f2-a0a2-7edd35f43899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:00.991609 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11525280-cdb0-46f2-a0a2-7edd35f43899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:00.992110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11525280-cdb0-46f2-a0a2-7edd35f43899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:00.992489 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11525280-cdb0-46f2-a0a2-7edd35f43899 admin admin] PATH_INFO: `/v3/projects/227ce65aaafa47a5be2d92f5813d0e74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:00.994010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11525280-cdb0-46f2-a0a2-7edd35f43899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=227ce65aaafa47a5be2d92f5813d0e74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:00.996344 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11525280-cdb0-46f2-a0a2-7edd35f43899 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:01.003523 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1233/2454] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:01.004751 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1e6ac-1af4-45df-8c7e-48a0a6bde8e9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:01.011755 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:01.035125 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1e6ac-1af4-45df-8c7e-48a0a6bde8e9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:01.045047 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14e0cb-0c8b-43dc-a47c-110830faf931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:01.046196 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14e0cb-0c8b-43dc-a47c-110830faf931 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:01.046668 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14e0cb-0c8b-43dc-a47c-110830faf931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:01.047128 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14e0cb-0c8b-43dc-a47c-110830faf931 admin admin] PATH_INFO: `/v3/projects/26b8b90155644c5f8382dc423eae8bf0/users/61564210ef43453794fe6333e2b661d1/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:01.050331 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14e0cb-0c8b-43dc-a47c-110830faf931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26b8b90155644c5f8382dc423eae8bf0, user_id=61564210ef43453794fe6333e2b661d1, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:01.054717 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14e0cb-0c8b-43dc-a47c-110830faf931 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:01.077457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1234/2455] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:53:01 2022] PUT /identity/v3/projects/26b8b90155644c5f8382dc423eae8bf0/users/61564210ef43453794fe6333e2b661d1/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:01.091247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e141b-0f9d-4711-a2bc-a4bfe5142fa5 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:01.091617 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e141b-0f9d-4711-a2bc-a4bfe5142fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:01.091925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e141b-0f9d-4711-a2bc-a4bfe5142fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:01.139244 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-387e141b-0f9d-4711-a2bc-a4bfe5142fa5 None None] MFA Rules not processed for user `61564210ef43453794fe6333e2b661d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:01.142689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-387e141b-0f9d-4711-a2bc-a4bfe5142fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:01.164986 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-11525280-cdb0-46f2-a0a2-7edd35f43899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '227ce65aaafa47a5be2d92f5813d0e74', 'request_id': 'req-11525280-cdb0-46f2-a0a2-7edd35f43899'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:01.176259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-11525280-cdb0-46f2-a0a2-7edd35f43899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '227ce65aaafa47a5be2d92f5813d0e74', 'request_id': 'req-11525280-cdb0-46f2-a0a2-7edd35f43899'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:01.189524 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-11525280-cdb0-46f2-a0a2-7edd35f43899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '227ce65aaafa47a5be2d92f5813d0e74', 'request_id': 'req-11525280-cdb0-46f2-a0a2-7edd35f43899'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:01.294667 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-11525280-cdb0-46f2-a0a2-7edd35f43899 admin admin] The token cache is being invalidate because project 227ce65aaafa47a5be2d92f5813d0e74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:53:01.294979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-11525280-cdb0-46f2-a0a2-7edd35f43899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:01.297574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-11525280-cdb0-46f2-a0a2-7edd35f43899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:01.299340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1222/2456] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:53:00 2022] DELETE /identity/v3/projects/227ce65aaafa47a5be2d92f5813d0e74 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:01.304833 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1235/2457] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:01 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:53:01.305359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d117d2-2368-4fbb-a68c-990e7840621f None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:01.305809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d117d2-2368-4fbb-a68c-990e7840621f None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:01.306095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d117d2-2368-4fbb-a68c-990e7840621f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:01.317047 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e141b-0f9d-4711-a2bc-a4bfe5142fa5 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:01.319740 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-387e141b-0f9d-4711-a2bc-a4bfe5142fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:01.356823 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-47d117d2-2368-4fbb-a68c-990e7840621f None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:01.358909 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-47d117d2-2368-4fbb-a68c-990e7840621f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:01.405057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e141b-0f9d-4711-a2bc-a4bfe5142fa5 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:01.415009 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe4dda-5730-4bea-a759-9af2f5b1d78a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:01.415712 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe4dda-5730-4bea-a759-9af2f5b1d78a service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:01.415988 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe4dda-5730-4bea-a759-9af2f5b1d78a service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:01.416259 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe4dda-5730-4bea-a759-9af2f5b1d78a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:01.420748 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6abe4dda-5730-4bea-a759-9af2f5b1d78a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:01.436385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe4dda-5730-4bea-a759-9af2f5b1d78a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:01.437809 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe4dda-5730-4bea-a759-9af2f5b1d78a service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:01.462151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1236/2458] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:01 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:53:01.463938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1223/2459] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:01.523222 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea705910-4a4f-47b7-bdd9-30a69d5e52a3 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:01.523520 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea705910-4a4f-47b7-bdd9-30a69d5e52a3 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:01.523840 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea705910-4a4f-47b7-bdd9-30a69d5e52a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:01.548709 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ea705910-4a4f-47b7-bdd9-30a69d5e52a3 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:01.550286 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ea705910-4a4f-47b7-bdd9-30a69d5e52a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:01.569379 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1237/2460] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:01.572849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d117d2-2368-4fbb-a68c-990e7840621f None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:01.586666 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d117d2-2368-4fbb-a68c-990e7840621f None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:01.619075 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713836b-0f1a-4347-af66-b1328ba3e18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:01.619847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713836b-0f1a-4347-af66-b1328ba3e18e admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:01.620131 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713836b-0f1a-4347-af66-b1328ba3e18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:01.620409 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713836b-0f1a-4347-af66-b1328ba3e18e admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:01.621268 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713836b-0f1a-4347-af66-b1328ba3e18e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:53:01.621624 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713836b-0f1a-4347-af66-b1328ba3e18e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:01.623271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713836b-0f1a-4347-af66-b1328ba3e18e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:01.655539 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1224/2461] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:53:01.663218 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea705910-4a4f-47b7-bdd9-30a69d5e52a3 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:01.679360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea705910-4a4f-47b7-bdd9-30a69d5e52a3 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:01.700848 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7224e34-8260-4d84-96f0-57816bb9c755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:01.716056 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7224e34-8260-4d84-96f0-57816bb9c755 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:01.718582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7224e34-8260-4d84-96f0-57816bb9c755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:01.718582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7224e34-8260-4d84-96f0-57816bb9c755 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:01.718582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7224e34-8260-4d84-96f0-57816bb9c755 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:01.720108 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7224e34-8260-4d84-96f0-57816bb9c755 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:01.865428 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1238/2462] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:53:01 2022] POST /identity/v3/projects => generated 407 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:01.869570 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713836b-0f1a-4347-af66-b1328ba3e18e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:01.883082 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713836b-0f1a-4347-af66-b1328ba3e18e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:01.964974 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb7214-bfdf-42b3-854f-1bd227970229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:01.965730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb7214-bfdf-42b3-854f-1bd227970229 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:01.966020 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb7214-bfdf-42b3-854f-1bd227970229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:01.966306 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb7214-bfdf-42b3-854f-1bd227970229 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:01.973004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb7214-bfdf-42b3-854f-1bd227970229 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:01.977314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb7214-bfdf-42b3-854f-1bd227970229 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:02.122481 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1225/2463] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:53:01 2022] POST /identity/v3/users => generated 304 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:02.123320 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7224e34-8260-4d84-96f0-57816bb9c755 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:02.136271 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7224e34-8260-4d84-96f0-57816bb9c755 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:02.143184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f90cf4-c2db-436f-b018-891cdc36f950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:02.146082 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f90cf4-c2db-436f-b018-891cdc36f950 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:02.146082 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f90cf4-c2db-436f-b018-891cdc36f950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:02.146082 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f90cf4-c2db-436f-b018-891cdc36f950 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:02.146082 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f90cf4-c2db-436f-b018-891cdc36f950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:02.147357 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f90cf4-c2db-436f-b018-891cdc36f950 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:02.158673 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1239/2464] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:02.163356 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb7214-bfdf-42b3-854f-1bd227970229 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:02.178119 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb7214-bfdf-42b3-854f-1bd227970229 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:02.185827 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7f895-851a-40fc-b443-a042fa188f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:02.186654 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7f895-851a-40fc-b443-a042fa188f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:02.186805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7f895-851a-40fc-b443-a042fa188f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:02.189015 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7f895-851a-40fc-b443-a042fa188f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:02.189166 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df7f895-851a-40fc-b443-a042fa188f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:02.190880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df7f895-851a-40fc-b443-a042fa188f7f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:02.204111 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1226/2465] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:02.209113 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f90cf4-c2db-436f-b018-891cdc36f950 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:02.220433 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f90cf4-c2db-436f-b018-891cdc36f950 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:02.225424 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834d077-8bff-4540-b116-b82b2d145a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:02.226497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834d077-8bff-4540-b116-b82b2d145a96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:02.226790 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834d077-8bff-4540-b116-b82b2d145a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:02.227068 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834d077-8bff-4540-b116-b82b2d145a96 admin admin] PATH_INFO: `/v3/projects/3bf24ee9f07441769880f966d35b2b7f/users/b52b4844172e4b8fad317f346a8d5f66/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:02.229712 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:02.245671 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834d077-8bff-4540-b116-b82b2d145a96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bf24ee9f07441769880f966d35b2b7f, user_id=b52b4844172e4b8fad317f346a8d5f66, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:02.249131 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4834d077-8bff-4540-b116-b82b2d145a96 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:02.327446 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1240/2466] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:53:02 2022] PUT /identity/v3/projects/3bf24ee9f07441769880f966d35b2b7f/users/b52b4844172e4b8fad317f346a8d5f66/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:53:02.345021 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84bcc8-fe65-445d-889d-8add7eeb4f61 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:02.345541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84bcc8-fe65-445d-889d-8add7eeb4f61 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:02.346054 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84bcc8-fe65-445d-889d-8add7eeb4f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:02.357784 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:02.387586 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-ee84bcc8-fe65-445d-889d-8add7eeb4f61 None None] MFA Rules not processed for user `b52b4844172e4b8fad317f346a8d5f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:02.387586 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ee84bcc8-fe65-445d-889d-8add7eeb4f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:02.471981 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1227/2467] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:02 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:02.477270 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834d077-8bff-4540-b116-b82b2d145a96 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:02.571330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834d077-8bff-4540-b116-b82b2d145a96 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:02.579312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca3fa6-4c26-4562-82d3-5e443d32e0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:02.580206 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca3fa6-4c26-4562-82d3-5e443d32e0cd admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:02.580578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca3fa6-4c26-4562-82d3-5e443d32e0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:02.580740 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca3fa6-4c26-4562-82d3-5e443d32e0cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:02.581716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aca3fa6-4c26-4562-82d3-5e443d32e0cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:02.583555 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aca3fa6-4c26-4562-82d3-5e443d32e0cd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:02.685775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1241/2468] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:53:02 2022] POST /identity/v3/projects => generated 409 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:02.690037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84bcc8-fe65-445d-889d-8add7eeb4f61 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:02.700819 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84bcc8-fe65-445d-889d-8add7eeb4f61 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:02.759425 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297a503-26fa-4d41-8207-206d56a4b882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:02.760331 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297a503-26fa-4d41-8207-206d56a4b882 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:02.760615 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297a503-26fa-4d41-8207-206d56a4b882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:02.760973 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297a503-26fa-4d41-8207-206d56a4b882 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:02.761851 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1297a503-26fa-4d41-8207-206d56a4b882 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:02.764031 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1297a503-26fa-4d41-8207-206d56a4b882 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:02.818874 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392d0bd-e668-4358-b332-d31851d13b5d None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:02.819170 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392d0bd-e668-4358-b332-d31851d13b5d None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:02.819457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392d0bd-e668-4358-b332-d31851d13b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:02.839364 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8392d0bd-e668-4358-b332-d31851d13b5d None None] MFA Rules not processed for user `ba4946e9565f4a608dd04fbc4de18dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:02.840624 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8392d0bd-e668-4358-b332-d31851d13b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:02.889898 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1228/2469] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:53:02 2022] POST /identity/v3/users => generated 311 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:02.894984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297a503-26fa-4d41-8207-206d56a4b882 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:02.911911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1242/2470] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:53:02.915654 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297a503-26fa-4d41-8207-206d56a4b882 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:02.931216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda60f7b-030b-4637-a20b-ff529ba290de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:02.932155 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda60f7b-030b-4637-a20b-ff529ba290de admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:02.932440 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda60f7b-030b-4637-a20b-ff529ba290de admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:02.932712 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda60f7b-030b-4637-a20b-ff529ba290de admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:02.933656 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda60f7b-030b-4637-a20b-ff529ba290de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:02.937957 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda60f7b-030b-4637-a20b-ff529ba290de admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:02.956133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1229/2471] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:02.959944 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392d0bd-e668-4358-b332-d31851d13b5d None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:02.979342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392d0bd-e668-4358-b332-d31851d13b5d None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:02.985999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed917908-0d84-48c5-921c-2c4d3e3e9423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:02.987187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed917908-0d84-48c5-921c-2c4d3e3e9423 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:02.987541 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed917908-0d84-48c5-921c-2c4d3e3e9423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:02.987890 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed917908-0d84-48c5-921c-2c4d3e3e9423 admin admin] PATH_INFO: `/v3/projects/e17d9e3419234e4885a79392e01f3bf2/users/0ffe75b1ef4a4c2d98e93335bfe66f17/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:03.008454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed917908-0d84-48c5-921c-2c4d3e3e9423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e17d9e3419234e4885a79392e01f3bf2, user_id=0ffe75b1ef4a4c2d98e93335bfe66f17, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:03.011934 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed917908-0d84-48c5-921c-2c4d3e3e9423 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:03.087595 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1243/2472] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:53:02 2022] PUT /identity/v3/projects/e17d9e3419234e4885a79392e01f3bf2/users/0ffe75b1ef4a4c2d98e93335bfe66f17/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:03.091754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda60f7b-030b-4637-a20b-ff529ba290de admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:03.107179 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda60f7b-030b-4637-a20b-ff529ba290de admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:03.416721 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947e1fb-bc75-48d7-9a06-8b73a1b76fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:03.416721 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947e1fb-bc75-48d7-9a06-8b73a1b76fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:03.416721 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947e1fb-bc75-48d7-9a06-8b73a1b76fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:03.416721 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947e1fb-bc75-48d7-9a06-8b73a1b76fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:03.418075 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947e1fb-bc75-48d7-9a06-8b73a1b76fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:03.418075 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947e1fb-bc75-48d7-9a06-8b73a1b76fc0 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:03.418075 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1230/2473] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:53:03.418505 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed917908-0d84-48c5-921c-2c4d3e3e9423 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:03.418505 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed917908-0d84-48c5-921c-2c4d3e3e9423 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:03.424080 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7edbd-bf42-4040-b95f-df03bdbb293e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:03.425089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7edbd-bf42-4040-b95f-df03bdbb293e admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:03.425469 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7edbd-bf42-4040-b95f-df03bdbb293e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:03.425809 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7edbd-bf42-4040-b95f-df03bdbb293e admin admin] PATH_INFO: `/v3/projects/e17d9e3419234e4885a79392e01f3bf2/users/0ffe75b1ef4a4c2d98e93335bfe66f17/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:03.428196 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7edbd-bf42-4040-b95f-df03bdbb293e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e17d9e3419234e4885a79392e01f3bf2, user_id=0ffe75b1ef4a4c2d98e93335bfe66f17, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:03.431191 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7edbd-bf42-4040-b95f-df03bdbb293e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:03.446503 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1244/2474] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:53:03 2022] PUT /identity/v3/projects/e17d9e3419234e4885a79392e01f3bf2/users/0ffe75b1ef4a4c2d98e93335bfe66f17/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:03.458267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19444647-41a6-465b-bfdf-e0b77343046a None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:03.458541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19444647-41a6-465b-bfdf-e0b77343046a None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:03.458834 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19444647-41a6-465b-bfdf-e0b77343046a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:03.489756 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-19444647-41a6-465b-bfdf-e0b77343046a None None] MFA Rules not processed for user `0ffe75b1ef4a4c2d98e93335bfe66f17`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:03.491000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-19444647-41a6-465b-bfdf-e0b77343046a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:03.577258 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1231/2475] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:03 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:03.588493 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7edbd-bf42-4040-b95f-df03bdbb293e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:03.590309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-02c7edbd-bf42-4040-b95f-df03bdbb293e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:03.641228 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:03.665084 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7edbd-bf42-4040-b95f-df03bdbb293e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:03.669261 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561ec3b-f821-471a-9e46-9060b063d753 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:03.669855 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561ec3b-f821-471a-9e46-9060b063d753 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:03.670168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561ec3b-f821-471a-9e46-9060b063d753 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:03.670477 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561ec3b-f821-471a-9e46-9060b063d753 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:03.676706 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19444647-41a6-465b-bfdf-e0b77343046a None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:03.678219 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-19444647-41a6-465b-bfdf-e0b77343046a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:03.682285 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561ec3b-f821-471a-9e46-9060b063d753 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:03.683818 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561ec3b-f821-471a-9e46-9060b063d753 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:03.695417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1245/2476] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:03 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:53:03.744275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19444647-41a6-465b-bfdf-e0b77343046a None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:03.754437 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5d61f-3c38-479a-85e5-b8a9fe603795 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:03.754719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5d61f-3c38-479a-85e5-b8a9fe603795 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:03.755013 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5d61f-3c38-479a-85e5-b8a9fe603795 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:03.755261 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5d61f-3c38-479a-85e5-b8a9fe603795 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:03.757193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-50f5d61f-3c38-479a-85e5-b8a9fe603795 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:03.768310 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5d61f-3c38-479a-85e5-b8a9fe603795 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:03.769741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5d61f-3c38-479a-85e5-b8a9fe603795 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:03.779901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1232/2477] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:53:04.627780 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:04.637185 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561ec3b-f821-471a-9e46-9060b063d753 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:04.647195 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561ec3b-f821-471a-9e46-9060b063d753 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:04.651160 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b470fa8-ab26-4fce-8afa-54cede945fef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:04.652013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b470fa8-ab26-4fce-8afa-54cede945fef service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:04.652184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b470fa8-ab26-4fce-8afa-54cede945fef service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:04.652464 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b470fa8-ab26-4fce-8afa-54cede945fef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:04.721757 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b470fa8-ab26-4fce-8afa-54cede945fef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:04.723394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b470fa8-ab26-4fce-8afa-54cede945fef service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:04.736777 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1246/2478] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:04 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:53:06.675738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5d61f-3c38-479a-85e5-b8a9fe603795 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:06.677207 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-50f5d61f-3c38-479a-85e5-b8a9fe603795 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:06.762852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5d61f-3c38-479a-85e5-b8a9fe603795 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:06.768704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1dabcb-5745-4739-9c60-bbca5f2ad912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:06.769774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dabcb-5745-4739-9c60-bbca5f2ad912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:06.770059 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dabcb-5745-4739-9c60-bbca5f2ad912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:06.771858 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dabcb-5745-4739-9c60-bbca5f2ad912 admin admin] PATH_INFO: `/v3/users/7780be00665642f291b24f28342f7d82` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:06.772662 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1dabcb-5745-4739-9c60-bbca5f2ad912 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7780be00665642f291b24f28342f7d82)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:06.774689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1dabcb-5745-4739-9c60-bbca5f2ad912 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:07.022828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:07.045034 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1a1dabcb-5745-4739-9c60-bbca5f2ad912 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7780be00665642f291b24f28342f7d82'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:07.046132 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1a1dabcb-5745-4739-9c60-bbca5f2ad912 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7780be00665642f291b24f28342f7d82'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:07.047775 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1a1dabcb-5745-4739-9c60-bbca5f2ad912 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7780be00665642f291b24f28342f7d82'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:07.066777 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1a1dabcb-5745-4739-9c60-bbca5f2ad912 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7780be00665642f291b24f28342f7d82'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:07.080197 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1a1dabcb-5745-4739-9c60-bbca5f2ad912 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7780be00665642f291b24f28342f7d82'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:07.200525 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1233/2479] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:53:06 2022] DELETE /identity/v3/users/7780be00665642f291b24f28342f7d82 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:07.200730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5839f4a-3d96-4f51-90c7-bb5b99fe4f15 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:07.200730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5839f4a-3d96-4f51-90c7-bb5b99fe4f15 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:07.200730 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5839f4a-3d96-4f51-90c7-bb5b99fe4f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:07.217315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d5839f4a-3d96-4f51-90c7-bb5b99fe4f15 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:07.218889 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d5839f4a-3d96-4f51-90c7-bb5b99fe4f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:07.364095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1247/2480] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:53:07.374133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1dabcb-5745-4739-9c60-bbca5f2ad912 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:07.376308 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1a1dabcb-5745-4739-9c60-bbca5f2ad912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:07.400351 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efc4f9-a3ea-4d64-8510-3927b5aaa794 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:07.401012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efc4f9-a3ea-4d64-8510-3927b5aaa794 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:07.401491 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efc4f9-a3ea-4d64-8510-3927b5aaa794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:07.440061 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a6efc4f9-a3ea-4d64-8510-3927b5aaa794 None None] MFA Rules not processed for user `2b03f8b8c51e472bbd4cf0afc9c6b632`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:07.444779 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a6efc4f9-a3ea-4d64-8510-3927b5aaa794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:07.494942 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1dabcb-5745-4739-9c60-bbca5f2ad912 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:07.499035 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a075859-3926-44ae-9930-b84840c4f8f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:07.499792 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a075859-3926-44ae-9930-b84840c4f8f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:07.500068 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a075859-3926-44ae-9930-b84840c4f8f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:07.500350 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a075859-3926-44ae-9930-b84840c4f8f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:07.513017 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a075859-3926-44ae-9930-b84840c4f8f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:07.513569 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a075859-3926-44ae-9930-b84840c4f8f9 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:07.525761 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1234/2481] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:53:07.538048 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1248/2482] 10.222.0.57 () {58 vars in 967 bytes} [Mon Jul 18 08:53:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:07.553772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a075859-3926-44ae-9930-b84840c4f8f9 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:07.555148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3a075859-3926-44ae-9930-b84840c4f8f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:07.636522 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a075859-3926-44ae-9930-b84840c4f8f9 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:07.643108 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f83c2-2459-4079-bade-f859cfc0633e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09a5c2e804504038817cad76544dce5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:07.644057 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f83c2-2459-4079-bade-f859cfc0633e service swift] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:07.644341 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f83c2-2459-4079-bade-f859cfc0633e service swift] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:07.644613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f83c2-2459-4079-bade-f859cfc0633e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:07.653919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f83c2-2459-4079-bade-f859cfc0633e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:07.655590 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f83c2-2459-4079-bade-f859cfc0633e service swift] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:07.665194 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1235/2483] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Jul 18 08:53:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 08:53:08.237142 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6efc4f9-a3ea-4d64-8510-3927b5aaa794 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:08.239550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a6efc4f9-a3ea-4d64-8510-3927b5aaa794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:08.292500 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6efc4f9-a3ea-4d64-8510-3927b5aaa794 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:08.296287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58286d3b-8c17-4c56-900d-a80d7b727537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:08.297881 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58286d3b-8c17-4c56-900d-a80d7b727537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:08.298172 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58286d3b-8c17-4c56-900d-a80d7b727537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:08.298452 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58286d3b-8c17-4c56-900d-a80d7b727537 admin admin] PATH_INFO: `/v3/projects/e4dfc36cfd8c4019970a277cb3935d38` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:08.300063 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58286d3b-8c17-4c56-900d-a80d7b727537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4dfc36cfd8c4019970a277cb3935d38)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:08.302073 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58286d3b-8c17-4c56-900d-a80d7b727537 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:08.333659 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:08.555434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-58286d3b-8c17-4c56-900d-a80d7b727537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4dfc36cfd8c4019970a277cb3935d38', 'request_id': 'req-58286d3b-8c17-4c56-900d-a80d7b727537'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:08.569162 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-58286d3b-8c17-4c56-900d-a80d7b727537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4dfc36cfd8c4019970a277cb3935d38', 'request_id': 'req-58286d3b-8c17-4c56-900d-a80d7b727537'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:08.579038 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-58286d3b-8c17-4c56-900d-a80d7b727537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4dfc36cfd8c4019970a277cb3935d38', 'request_id': 'req-58286d3b-8c17-4c56-900d-a80d7b727537'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:08.657887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-58286d3b-8c17-4c56-900d-a80d7b727537 admin admin] The token cache is being invalidate because project e4dfc36cfd8c4019970a277cb3935d38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:53:08.658185 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-58286d3b-8c17-4c56-900d-a80d7b727537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:08.658825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-58286d3b-8c17-4c56-900d-a80d7b727537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:08.663825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1249/2484] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:53:08 2022] DELETE /identity/v3/projects/e4dfc36cfd8c4019970a277cb3935d38 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:08.665239 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f83c2-2459-4079-bade-f859cfc0633e service swift] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:08.677109 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f83c2-2459-4079-bade-f859cfc0633e service swift] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:08.739971 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:08.752411 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73899996-481d-4902-9381-26bfea68c7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:08.753583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73899996-481d-4902-9381-26bfea68c7dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:08.754135 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73899996-481d-4902-9381-26bfea68c7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:08.754135 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73899996-481d-4902-9381-26bfea68c7dd admin admin] PATH_INFO: `/v3/users/61564210ef43453794fe6333e2b661d1` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:08.756023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73899996-481d-4902-9381-26bfea68c7dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=61564210ef43453794fe6333e2b661d1)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:08.758604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73899996-481d-4902-9381-26bfea68c7dd admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:09.092520 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-73899996-481d-4902-9381-26bfea68c7dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61564210ef43453794fe6333e2b661d1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:09.093876 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-73899996-481d-4902-9381-26bfea68c7dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61564210ef43453794fe6333e2b661d1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:09.094511 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-73899996-481d-4902-9381-26bfea68c7dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61564210ef43453794fe6333e2b661d1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:09.107176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-73899996-481d-4902-9381-26bfea68c7dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61564210ef43453794fe6333e2b661d1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:09.114580 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-73899996-481d-4902-9381-26bfea68c7dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61564210ef43453794fe6333e2b661d1'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:09.204254 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1236/2485] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:53:08 2022] DELETE /identity/v3/users/61564210ef43453794fe6333e2b661d1 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:09.746813 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58286d3b-8c17-4c56-900d-a80d7b727537 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:09.748547 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-58286d3b-8c17-4c56-900d-a80d7b727537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:09.764642 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58286d3b-8c17-4c56-900d-a80d7b727537 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:09.856912 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f457e-b347-422a-9aa8-03245f4ba543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:09.857788 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f457e-b347-422a-9aa8-03245f4ba543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:09.858079 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f457e-b347-422a-9aa8-03245f4ba543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:09.858359 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f457e-b347-422a-9aa8-03245f4ba543 admin admin] PATH_INFO: `/v3/projects/26b8b90155644c5f8382dc423eae8bf0` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:09.861068 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f457e-b347-422a-9aa8-03245f4ba543 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26b8b90155644c5f8382dc423eae8bf0)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:09.863020 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f457e-b347-422a-9aa8-03245f4ba543 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:09.973707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1a0f457e-b347-422a-9aa8-03245f4ba543 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26b8b90155644c5f8382dc423eae8bf0', 'request_id': 'req-1a0f457e-b347-422a-9aa8-03245f4ba543'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:09.979716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1a0f457e-b347-422a-9aa8-03245f4ba543 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26b8b90155644c5f8382dc423eae8bf0', 'request_id': 'req-1a0f457e-b347-422a-9aa8-03245f4ba543'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:09.986157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1a0f457e-b347-422a-9aa8-03245f4ba543 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26b8b90155644c5f8382dc423eae8bf0', 'request_id': 'req-1a0f457e-b347-422a-9aa8-03245f4ba543'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:10.058463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1a0f457e-b347-422a-9aa8-03245f4ba543 admin admin] The token cache is being invalidate because project 26b8b90155644c5f8382dc423eae8bf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:53:10.058899 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1a0f457e-b347-422a-9aa8-03245f4ba543 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:10.059644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-1a0f457e-b347-422a-9aa8-03245f4ba543 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:10.061440 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1250/2486] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:53:09 2022] DELETE /identity/v3/projects/26b8b90155644c5f8382dc423eae8bf0 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:53:10.067070 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dc1c6-4119-4058-9232-161d2ba597d1 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:10.067771 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dc1c6-4119-4058-9232-161d2ba597d1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:10.067771 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dc1c6-4119-4058-9232-161d2ba597d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:10.087445 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-5d2dc1c6-4119-4058-9232-161d2ba597d1 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:10.088720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5d2dc1c6-4119-4058-9232-161d2ba597d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:10.150717 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1237/2487] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:53:10.160440 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da84e0-7500-49e4-87cb-b7aaafa6620e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:10.160817 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da84e0-7500-49e4-87cb-b7aaafa6620e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:10.161165 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da84e0-7500-49e4-87cb-b7aaafa6620e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:10.181377 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-38da84e0-7500-49e4-87cb-b7aaafa6620e None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:10.182580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-38da84e0-7500-49e4-87cb-b7aaafa6620e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:10.191184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1251/2488] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:10.196286 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dc1c6-4119-4058-9232-161d2ba597d1 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:10.206936 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dc1c6-4119-4058-9232-161d2ba597d1 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:10.211826 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784c004-9ef0-4a0a-b33d-50bb7eb77887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:10.212530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784c004-9ef0-4a0a-b33d-50bb7eb77887 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:10.212862 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784c004-9ef0-4a0a-b33d-50bb7eb77887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:10.213141 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784c004-9ef0-4a0a-b33d-50bb7eb77887 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:10.213878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5784c004-9ef0-4a0a-b33d-50bb7eb77887 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:53:10.214187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5784c004-9ef0-4a0a-b33d-50bb7eb77887 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:10.215814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5784c004-9ef0-4a0a-b33d-50bb7eb77887 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:10.228901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1238/2489] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:53:10.234015 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da84e0-7500-49e4-87cb-b7aaafa6620e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:10.244397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da84e0-7500-49e4-87cb-b7aaafa6620e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:10.249304 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c5695-a5ce-4f43-a7ba-13c913b8d554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:10.250362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c5695-a5ce-4f43-a7ba-13c913b8d554 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:10.250640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c5695-a5ce-4f43-a7ba-13c913b8d554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:10.250967 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c5695-a5ce-4f43-a7ba-13c913b8d554 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:10.251723 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c5695-a5ce-4f43-a7ba-13c913b8d554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:10.253689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c5695-a5ce-4f43-a7ba-13c913b8d554 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:10.380185 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1252/2490] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:53:10 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:10.384492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784c004-9ef0-4a0a-b33d-50bb7eb77887 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:10.395838 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784c004-9ef0-4a0a-b33d-50bb7eb77887 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:10.450192 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:10.457522 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6609a-3c84-420e-817b-5cff21123e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:10.458198 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6609a-3c84-420e-817b-5cff21123e3e admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:10.461880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6609a-3c84-420e-817b-5cff21123e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:10.461880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6609a-3c84-420e-817b-5cff21123e3e admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:10.461880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6609a-3c84-420e-817b-5cff21123e3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:10.461880 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6609a-3c84-420e-817b-5cff21123e3e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:10.578966 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1239/2491] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:53:10 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:10.584105 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c5695-a5ce-4f43-a7ba-13c913b8d554 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:10.596631 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c5695-a5ce-4f43-a7ba-13c913b8d554 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:10.601576 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b0966-0425-4b60-8da2-a27f41f4085b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:10.602238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b0966-0425-4b60-8da2-a27f41f4085b admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:10.602682 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b0966-0425-4b60-8da2-a27f41f4085b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:10.602963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b0966-0425-4b60-8da2-a27f41f4085b admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:10.603734 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b0966-0425-4b60-8da2-a27f41f4085b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:10.605509 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b0966-0425-4b60-8da2-a27f41f4085b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:10.607159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:10.618900 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1253/2492] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:10.622865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6609a-3c84-420e-817b-5cff21123e3e admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:10.633846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6609a-3c84-420e-817b-5cff21123e3e admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:10.638752 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53607e65-aeb5-4f4b-8299-5efe318609b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:10.639492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53607e65-aeb5-4f4b-8299-5efe318609b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:10.639740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53607e65-aeb5-4f4b-8299-5efe318609b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:10.640001 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53607e65-aeb5-4f4b-8299-5efe318609b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:10.640773 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53607e65-aeb5-4f4b-8299-5efe318609b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:10.642359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53607e65-aeb5-4f4b-8299-5efe318609b3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:10.657136 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1240/2493] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:53:10.659012 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b0966-0425-4b60-8da2-a27f41f4085b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:10.669931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b0966-0425-4b60-8da2-a27f41f4085b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:10.677595 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcfcc8d-3b93-4d3c-98b5-4c25fd2f6e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:10.678562 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcfcc8d-3b93-4d3c-98b5-4c25fd2f6e7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:10.679860 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcfcc8d-3b93-4d3c-98b5-4c25fd2f6e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:10.679860 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcfcc8d-3b93-4d3c-98b5-4c25fd2f6e7f admin admin] PATH_INFO: `/v3/projects/025abe7868084bf7acf06f510bd021b5/users/4e4e1ed088d54ba6a917d02d90283d69/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:10.695399 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcfcc8d-3b93-4d3c-98b5-4c25fd2f6e7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=025abe7868084bf7acf06f510bd021b5, user_id=4e4e1ed088d54ba6a917d02d90283d69, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:10.698088 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcfcc8d-3b93-4d3c-98b5-4c25fd2f6e7f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:10.790716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1254/2494] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:53:10 2022] PUT /identity/v3/projects/025abe7868084bf7acf06f510bd021b5/users/4e4e1ed088d54ba6a917d02d90283d69/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:10.802069 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382868c-5a46-48bd-8880-f90fdea1289a None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:10.802368 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382868c-5a46-48bd-8880-f90fdea1289a None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:10.802652 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382868c-5a46-48bd-8880-f90fdea1289a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:10.838932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2382868c-5a46-48bd-8880-f90fdea1289a None None] MFA Rules not processed for user `4e4e1ed088d54ba6a917d02d90283d69`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:10.841686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2382868c-5a46-48bd-8880-f90fdea1289a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:10.941021 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1241/2495] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:10.950063 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcfcc8d-3b93-4d3c-98b5-4c25fd2f6e7f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:11.055718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcfcc8d-3b93-4d3c-98b5-4c25fd2f6e7f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:11.061122 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cb49a-8524-482c-9859-206e87e4b1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:11.061783 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cb49a-8524-482c-9859-206e87e4b1c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:11.062136 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cb49a-8524-482c-9859-206e87e4b1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:11.063052 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cb49a-8524-482c-9859-206e87e4b1c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:11.064003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8cb49a-8524-482c-9859-206e87e4b1c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:11.066820 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8cb49a-8524-482c-9859-206e87e4b1c9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:11.184296 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1255/2496] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:53:10 2022] POST /identity/v3/projects => generated 377 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:11.189443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382868c-5a46-48bd-8880-f90fdea1289a None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:11.203752 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382868c-5a46-48bd-8880-f90fdea1289a None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:11.283644 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93526c-e938-447f-80de-e72a72b861ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:11.284734 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93526c-e938-447f-80de-e72a72b861ca admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:11.285220 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93526c-e938-447f-80de-e72a72b861ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:11.285497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93526c-e938-447f-80de-e72a72b861ca admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:11.286217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93526c-e938-447f-80de-e72a72b861ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:11.288362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93526c-e938-447f-80de-e72a72b861ca admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:11.414913 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1242/2497] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:53:11 2022] POST /identity/v3/users => generated 295 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:11.419181 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cb49a-8524-482c-9859-206e87e4b1c9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:11.430841 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cb49a-8524-482c-9859-206e87e4b1c9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:11.436169 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d943ef6-23e9-4a2f-88e6-b6df2862041f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:11.440775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d943ef6-23e9-4a2f-88e6-b6df2862041f admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:11.440775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d943ef6-23e9-4a2f-88e6-b6df2862041f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:11.440775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d943ef6-23e9-4a2f-88e6-b6df2862041f admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:11.440775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d943ef6-23e9-4a2f-88e6-b6df2862041f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:11.440775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d943ef6-23e9-4a2f-88e6-b6df2862041f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:11.452204 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1256/2498] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:11.457351 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93526c-e938-447f-80de-e72a72b861ca admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:11.467761 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93526c-e938-447f-80de-e72a72b861ca admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:11.473231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafd459-6d96-4996-8622-aa95f24d0fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:11.474210 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafd459-6d96-4996-8622-aa95f24d0fc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:11.474498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafd459-6d96-4996-8622-aa95f24d0fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:11.474771 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafd459-6d96-4996-8622-aa95f24d0fc9 admin admin] PATH_INFO: `/v3/projects/e97e66f048d2408cb7a5d08bab724c3c/users/c17c83414cfd4eb599f8fff5afe73a3a/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:11.491237 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bafd459-6d96-4996-8622-aa95f24d0fc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e97e66f048d2408cb7a5d08bab724c3c, user_id=c17c83414cfd4eb599f8fff5afe73a3a, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:11.494044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bafd459-6d96-4996-8622-aa95f24d0fc9 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:11.603915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1243/2499] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:53:11 2022] PUT /identity/v3/projects/e97e66f048d2408cb7a5d08bab724c3c/users/c17c83414cfd4eb599f8fff5afe73a3a/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:11.608177 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d943ef6-23e9-4a2f-88e6-b6df2862041f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:11.619481 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d943ef6-23e9-4a2f-88e6-b6df2862041f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:11.689854 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674420e-c666-497d-bf52-484f61820319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:11.691626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674420e-c666-497d-bf52-484f61820319 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:11.691626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674420e-c666-497d-bf52-484f61820319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:11.691626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674420e-c666-497d-bf52-484f61820319 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:11.692254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674420e-c666-497d-bf52-484f61820319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:11.694141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674420e-c666-497d-bf52-484f61820319 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:11.710823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1257/2500] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:11.714981 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafd459-6d96-4996-8622-aa95f24d0fc9 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:11.729664 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafd459-6d96-4996-8622-aa95f24d0fc9 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:11.737367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8e3c6-21bf-4041-bf09-c1a8c929dbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:11.739245 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8e3c6-21bf-4041-bf09-c1a8c929dbd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:11.744366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8e3c6-21bf-4041-bf09-c1a8c929dbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:11.744366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8e3c6-21bf-4041-bf09-c1a8c929dbd8 admin admin] PATH_INFO: `/v3/projects/e97e66f048d2408cb7a5d08bab724c3c/users/c17c83414cfd4eb599f8fff5afe73a3a/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:11.744366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8e3c6-21bf-4041-bf09-c1a8c929dbd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e97e66f048d2408cb7a5d08bab724c3c, user_id=c17c83414cfd4eb599f8fff5afe73a3a, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:11.746391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8e3c6-21bf-4041-bf09-c1a8c929dbd8 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:11.764961 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1244/2501] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:53:11 2022] PUT /identity/v3/projects/e97e66f048d2408cb7a5d08bab724c3c/users/c17c83414cfd4eb599f8fff5afe73a3a/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:11.782251 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83459a-5379-4c36-9102-e1effd297600 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:11.782251 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83459a-5379-4c36-9102-e1effd297600 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:11.782251 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83459a-5379-4c36-9102-e1effd297600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:11.849920 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-9c83459a-5379-4c36-9102-e1effd297600 None None] MFA Rules not processed for user `c17c83414cfd4eb599f8fff5afe73a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:11.851234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9c83459a-5379-4c36-9102-e1effd297600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:11.885942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:11.942503 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1258/2502] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:12.120694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8e3c6-21bf-4041-bf09-c1a8c929dbd8 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:12.124234 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ada8e3c6-21bf-4041-bf09-c1a8c929dbd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:12.157840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:12.223947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8e3c6-21bf-4041-bf09-c1a8c929dbd8 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:12.228847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7eeeec-f717-4b0e-b27b-73d1d0003948 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:12.229545 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7eeeec-f717-4b0e-b27b-73d1d0003948 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:12.229912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7eeeec-f717-4b0e-b27b-73d1d0003948 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:12.230205 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7eeeec-f717-4b0e-b27b-73d1d0003948 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:12.315493 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7eeeec-f717-4b0e-b27b-73d1d0003948 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:12.317132 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7eeeec-f717-4b0e-b27b-73d1d0003948 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:12.329578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1245/2503] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:53:16.122723 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83459a-5379-4c36-9102-e1effd297600 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:16.124157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9c83459a-5379-4c36-9102-e1effd297600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:16.190683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83459a-5379-4c36-9102-e1effd297600 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:16.196189 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788684e-3f00-485d-9eea-9d710e84f421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:16.197063 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788684e-3f00-485d-9eea-9d710e84f421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:16.197343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788684e-3f00-485d-9eea-9d710e84f421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:16.197612 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788684e-3f00-485d-9eea-9d710e84f421 admin admin] PATH_INFO: `/v3/users/b52b4844172e4b8fad317f346a8d5f66` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:16.198807 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788684e-3f00-485d-9eea-9d710e84f421 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b52b4844172e4b8fad317f346a8d5f66)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:16.200773 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788684e-3f00-485d-9eea-9d710e84f421 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:16.429738 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3788684e-3f00-485d-9eea-9d710e84f421 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b52b4844172e4b8fad317f346a8d5f66'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:16.430423 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3788684e-3f00-485d-9eea-9d710e84f421 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b52b4844172e4b8fad317f346a8d5f66'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:16.431039 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3788684e-3f00-485d-9eea-9d710e84f421 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b52b4844172e4b8fad317f346a8d5f66'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:16.444734 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3788684e-3f00-485d-9eea-9d710e84f421 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b52b4844172e4b8fad317f346a8d5f66'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:16.452014 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3788684e-3f00-485d-9eea-9d710e84f421 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b52b4844172e4b8fad317f346a8d5f66'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:16.579347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1259/2504] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:53:16 2022] DELETE /identity/v3/users/b52b4844172e4b8fad317f346a8d5f66 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:16.584427 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c31b2-cba6-4da7-8202-24ed20862bdb None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:16.584979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c31b2-cba6-4da7-8202-24ed20862bdb None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:16.584979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c31b2-cba6-4da7-8202-24ed20862bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:16.605081 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a92c31b2-cba6-4da7-8202-24ed20862bdb None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:16.606314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a92c31b2-cba6-4da7-8202-24ed20862bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:16.669213 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1246/2505] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:53:16.679162 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788684e-3f00-485d-9eea-9d710e84f421 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:16.680637 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3788684e-3f00-485d-9eea-9d710e84f421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:16.739087 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:16.744384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788684e-3f00-485d-9eea-9d710e84f421 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:16.748075 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bafed7-be15-4e84-a3d9-5c4b8c8d31d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:16.748726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bafed7-be15-4e84-a3d9-5c4b8c8d31d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:16.749009 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bafed7-be15-4e84-a3d9-5c4b8c8d31d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:16.749284 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bafed7-be15-4e84-a3d9-5c4b8c8d31d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:16.760404 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bafed7-be15-4e84-a3d9-5c4b8c8d31d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:16.761851 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bafed7-be15-4e84-a3d9-5c4b8c8d31d3 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:16.773161 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1260/2506] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:53:17.273930 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c31b2-cba6-4da7-8202-24ed20862bdb None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:17.277996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a92c31b2-cba6-4da7-8202-24ed20862bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:17.290247 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c31b2-cba6-4da7-8202-24ed20862bdb None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:17.294967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45bf420-292e-4ee9-a1ce-352bd8bfd858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:17.296602 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45bf420-292e-4ee9-a1ce-352bd8bfd858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:17.296857 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45bf420-292e-4ee9-a1ce-352bd8bfd858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:17.297134 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45bf420-292e-4ee9-a1ce-352bd8bfd858 admin admin] PATH_INFO: `/v3/projects/3bf24ee9f07441769880f966d35b2b7f` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:17.298353 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45bf420-292e-4ee9-a1ce-352bd8bfd858 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bf24ee9f07441769880f966d35b2b7f)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:17.300393 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45bf420-292e-4ee9-a1ce-352bd8bfd858 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:17.422635 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c45bf420-292e-4ee9-a1ce-352bd8bfd858 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bf24ee9f07441769880f966d35b2b7f', 'request_id': 'req-c45bf420-292e-4ee9-a1ce-352bd8bfd858'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:17.430908 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c45bf420-292e-4ee9-a1ce-352bd8bfd858 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bf24ee9f07441769880f966d35b2b7f', 'request_id': 'req-c45bf420-292e-4ee9-a1ce-352bd8bfd858'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:17.437751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c45bf420-292e-4ee9-a1ce-352bd8bfd858 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bf24ee9f07441769880f966d35b2b7f', 'request_id': 'req-c45bf420-292e-4ee9-a1ce-352bd8bfd858'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:17.440491 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:17.549210 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c45bf420-292e-4ee9-a1ce-352bd8bfd858 admin admin] The token cache is being invalidate because project 3bf24ee9f07441769880f966d35b2b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:53:17.549503 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c45bf420-292e-4ee9-a1ce-352bd8bfd858 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:17.550174 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c45bf420-292e-4ee9-a1ce-352bd8bfd858 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:17.551762 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1247/2507] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:53:17 2022] DELETE /identity/v3/projects/3bf24ee9f07441769880f966d35b2b7f => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:17.556341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bafed7-be15-4e84-a3d9-5c4b8c8d31d3 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:17.566239 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bafed7-be15-4e84-a3d9-5c4b8c8d31d3 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:17.627133 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d74f69-554a-44ca-8734-0db299738e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:17.628059 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d74f69-554a-44ca-8734-0db299738e63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:17.628423 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d74f69-554a-44ca-8734-0db299738e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:17.628721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d74f69-554a-44ca-8734-0db299738e63 admin admin] PATH_INFO: `/v3/users/0ffe75b1ef4a4c2d98e93335bfe66f17` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:17.630089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d74f69-554a-44ca-8734-0db299738e63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ffe75b1ef4a4c2d98e93335bfe66f17)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:17.632951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d74f69-554a-44ca-8734-0db299738e63 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:17.971099 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-57d74f69-554a-44ca-8734-0db299738e63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ffe75b1ef4a4c2d98e93335bfe66f17'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:17.972073 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-57d74f69-554a-44ca-8734-0db299738e63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ffe75b1ef4a4c2d98e93335bfe66f17'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:17.972426 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-57d74f69-554a-44ca-8734-0db299738e63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ffe75b1ef4a4c2d98e93335bfe66f17'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:17.988339 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-57d74f69-554a-44ca-8734-0db299738e63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ffe75b1ef4a4c2d98e93335bfe66f17'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:17.998289 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-57d74f69-554a-44ca-8734-0db299738e63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ffe75b1ef4a4c2d98e93335bfe66f17'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:18.096978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1261/2508] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:53:17 2022] DELETE /identity/v3/users/0ffe75b1ef4a4c2d98e93335bfe66f17 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:18.605829 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45bf420-292e-4ee9-a1ce-352bd8bfd858 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:18.606757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c45bf420-292e-4ee9-a1ce-352bd8bfd858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:18.625952 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45bf420-292e-4ee9-a1ce-352bd8bfd858 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:18.688287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299cba6c-6d54-4bcb-b8d3-e67827c2faa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:18.689179 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299cba6c-6d54-4bcb-b8d3-e67827c2faa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:18.689457 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299cba6c-6d54-4bcb-b8d3-e67827c2faa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:18.689733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299cba6c-6d54-4bcb-b8d3-e67827c2faa6 admin admin] PATH_INFO: `/v3/projects/e17d9e3419234e4885a79392e01f3bf2` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:18.690945 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299cba6c-6d54-4bcb-b8d3-e67827c2faa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e17d9e3419234e4885a79392e01f3bf2)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:18.693019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299cba6c-6d54-4bcb-b8d3-e67827c2faa6 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:18.828241 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-299cba6c-6d54-4bcb-b8d3-e67827c2faa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e17d9e3419234e4885a79392e01f3bf2', 'request_id': 'req-299cba6c-6d54-4bcb-b8d3-e67827c2faa6'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:18.838902 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-299cba6c-6d54-4bcb-b8d3-e67827c2faa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e17d9e3419234e4885a79392e01f3bf2', 'request_id': 'req-299cba6c-6d54-4bcb-b8d3-e67827c2faa6'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:18.842045 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-299cba6c-6d54-4bcb-b8d3-e67827c2faa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e17d9e3419234e4885a79392e01f3bf2', 'request_id': 'req-299cba6c-6d54-4bcb-b8d3-e67827c2faa6'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:18.975280 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-299cba6c-6d54-4bcb-b8d3-e67827c2faa6 admin admin] The token cache is being invalidate because project e17d9e3419234e4885a79392e01f3bf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:53:18.975788 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-299cba6c-6d54-4bcb-b8d3-e67827c2faa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:18.976687 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-299cba6c-6d54-4bcb-b8d3-e67827c2faa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:18.978386 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1248/2509] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:53:18 2022] DELETE /identity/v3/projects/e17d9e3419234e4885a79392e01f3bf2 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:53:18.984375 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae9a18-fd02-4026-bc43-1a40357aa111 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:18.984375 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae9a18-fd02-4026-bc43-1a40357aa111 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:18.984375 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae9a18-fd02-4026-bc43-1a40357aa111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:19.002794 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-67ae9a18-fd02-4026-bc43-1a40357aa111 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:19.004079 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-67ae9a18-fd02-4026-bc43-1a40357aa111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:19.038335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299cba6c-6d54-4bcb-b8d3-e67827c2faa6 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:19.039938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-299cba6c-6d54-4bcb-b8d3-e67827c2faa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:19.074453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1262/2510] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:19.082515 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844c31e-85ee-42e1-8a94-9c2bbddeed7e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:19.082810 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844c31e-85ee-42e1-8a94-9c2bbddeed7e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:19.083102 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844c31e-85ee-42e1-8a94-9c2bbddeed7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:19.102030 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-f844c31e-85ee-42e1-8a94-9c2bbddeed7e None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:19.103235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-f844c31e-85ee-42e1-8a94-9c2bbddeed7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:19.104099 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299cba6c-6d54-4bcb-b8d3-e67827c2faa6 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:19.107927 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46407068-cbbd-4c96-a92c-71d09e569bce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:19.108582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46407068-cbbd-4c96-a92c-71d09e569bce service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:19.108869 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46407068-cbbd-4c96-a92c-71d09e569bce service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:19.109149 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46407068-cbbd-4c96-a92c-71d09e569bce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:19.116331 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-46407068-cbbd-4c96-a92c-71d09e569bce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:19.116522 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1263/2511] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:53:19.116522 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844c31e-85ee-42e1-8a94-9c2bbddeed7e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:19.120516 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:19.125910 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844c31e-85ee-42e1-8a94-9c2bbddeed7e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:19.130694 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a0467-b313-437f-adb6-2d6bcae4252c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:19.132571 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a0467-b313-437f-adb6-2d6bcae4252c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:19.132861 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a0467-b313-437f-adb6-2d6bcae4252c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:19.133143 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a0467-b313-437f-adb6-2d6bcae4252c admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:19.133879 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a0467-b313-437f-adb6-2d6bcae4252c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:53:19.134195 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a0467-b313-437f-adb6-2d6bcae4252c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:19.135856 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a0467-b313-437f-adb6-2d6bcae4252c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:19.148063 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1264/2512] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:53:19.152372 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a0467-b313-437f-adb6-2d6bcae4252c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:19.156646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:19.163113 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a0467-b313-437f-adb6-2d6bcae4252c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:19.168028 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d58d0-7c4c-453d-8ef8-85161d2e1688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:19.168686 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d58d0-7c4c-453d-8ef8-85161d2e1688 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:19.168975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d58d0-7c4c-453d-8ef8-85161d2e1688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:19.169258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d58d0-7c4c-453d-8ef8-85161d2e1688 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:19.169954 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d58d0-7c4c-453d-8ef8-85161d2e1688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:19.171940 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d58d0-7c4c-453d-8ef8-85161d2e1688 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:19.183248 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46407068-cbbd-4c96-a92c-71d09e569bce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:19.184724 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46407068-cbbd-4c96-a92c-71d09e569bce service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:19.199410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1249/2513] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:53:19.285312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1265/2514] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:53:19 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:53:19.290315 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46407068-cbbd-4c96-a92c-71d09e569bce service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:19.300944 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46407068-cbbd-4c96-a92c-71d09e569bce service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:19.363136 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8fc3a-6ed8-4f76-a2f8-b9908f8a3376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:19.363885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8fc3a-6ed8-4f76-a2f8-b9908f8a3376 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:19.364169 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8fc3a-6ed8-4f76-a2f8-b9908f8a3376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:19.364447 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8fc3a-6ed8-4f76-a2f8-b9908f8a3376 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:19.365149 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8fc3a-6ed8-4f76-a2f8-b9908f8a3376 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:19.368078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8fc3a-6ed8-4f76-a2f8-b9908f8a3376 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:19.487089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1250/2515] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:53:19 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:19.492375 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d58d0-7c4c-453d-8ef8-85161d2e1688 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:19.505519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d58d0-7c4c-453d-8ef8-85161d2e1688 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:19.510885 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe3f2e-fa8f-4269-92a3-06cf6f5c1695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:19.511779 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe3f2e-fa8f-4269-92a3-06cf6f5c1695 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:19.512134 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe3f2e-fa8f-4269-92a3-06cf6f5c1695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:19.512419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe3f2e-fa8f-4269-92a3-06cf6f5c1695 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:19.513176 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe3f2e-fa8f-4269-92a3-06cf6f5c1695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:19.514782 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe3f2e-fa8f-4269-92a3-06cf6f5c1695 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:19.526021 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1266/2516] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:53:19.538119 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8fc3a-6ed8-4f76-a2f8-b9908f8a3376 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:19.549495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8fc3a-6ed8-4f76-a2f8-b9908f8a3376 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:19.555918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9a8b3-4c57-4456-a649-30a96f6db30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:19.557267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9a8b3-4c57-4456-a649-30a96f6db30d admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:19.557550 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9a8b3-4c57-4456-a649-30a96f6db30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:19.557995 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9a8b3-4c57-4456-a649-30a96f6db30d admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:19.558566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9a8b3-4c57-4456-a649-30a96f6db30d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:19.560246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9a8b3-4c57-4456-a649-30a96f6db30d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:19.571605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1251/2517] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:19.576018 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe3f2e-fa8f-4269-92a3-06cf6f5c1695 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:19.590863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe3f2e-fa8f-4269-92a3-06cf6f5c1695 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:19.596709 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a886a-7b45-48f2-99e5-3997478eda12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:19.597692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a886a-7b45-48f2-99e5-3997478eda12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:19.598282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a886a-7b45-48f2-99e5-3997478eda12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:19.598282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a886a-7b45-48f2-99e5-3997478eda12 admin admin] PATH_INFO: `/v3/projects/93f244b23dab47ca99a30773d83ffd46/users/ef46e5b7c97a4a8196476141700b0aef/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:19.618522 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a886a-7b45-48f2-99e5-3997478eda12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93f244b23dab47ca99a30773d83ffd46, user_id=ef46e5b7c97a4a8196476141700b0aef, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:19.621663 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a886a-7b45-48f2-99e5-3997478eda12 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:19.701428 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1267/2518] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:53:19 2022] PUT /identity/v3/projects/93f244b23dab47ca99a30773d83ffd46/users/ef46e5b7c97a4a8196476141700b0aef/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:19.712458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ef48c-64d2-4991-9fd6-d9af6e614b3d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:19.712999 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ef48c-64d2-4991-9fd6-d9af6e614b3d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:19.713127 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ef48c-64d2-4991-9fd6-d9af6e614b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:19.747498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-1e0ef48c-64d2-4991-9fd6-d9af6e614b3d None None] MFA Rules not processed for user `ef46e5b7c97a4a8196476141700b0aef`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:19.748703 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1e0ef48c-64d2-4991-9fd6-d9af6e614b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:19.821919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1252/2519] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:19.827400 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a886a-7b45-48f2-99e5-3997478eda12 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:19.899283 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a886a-7b45-48f2-99e5-3997478eda12 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:19.909579 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281f55e-f077-4b74-b0c2-d2775d205ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:19.910303 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281f55e-f077-4b74-b0c2-d2775d205ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:19.910616 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281f55e-f077-4b74-b0c2-d2775d205ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:19.910893 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281f55e-f077-4b74-b0c2-d2775d205ca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:19.911592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281f55e-f077-4b74-b0c2-d2775d205ca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:19.914629 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281f55e-f077-4b74-b0c2-d2775d205ca5 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:19.958195 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ef48c-64d2-4991-9fd6-d9af6e614b3d None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:19.960026 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1e0ef48c-64d2-4991-9fd6-d9af6e614b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:20.011598 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1268/2520] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:53:19 2022] POST /identity/v3/projects => generated 381 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:20.013285 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281f55e-f077-4b74-b0c2-d2775d205ca5 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:20.015858 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:20.027645 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ef48c-64d2-4991-9fd6-d9af6e614b3d None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:20.079157 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d456d-0d84-45ab-8488-e5c93478ecd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:20.079885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d456d-0d84-45ab-8488-e5c93478ecd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:20.080167 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d456d-0d84-45ab-8488-e5c93478ecd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:20.080449 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d456d-0d84-45ab-8488-e5c93478ecd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:20.082377 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-947d456d-0d84-45ab-8488-e5c93478ecd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:20.091584 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281f55e-f077-4b74-b0c2-d2775d205ca5 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:20.095866 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d456d-0d84-45ab-8488-e5c93478ecd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:20.097390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d456d-0d84-45ab-8488-e5c93478ecd9 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:20.098354 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c65c9b-d720-4cb6-9e3b-283bf66a9eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:20.099041 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c65c9b-d720-4cb6-9e3b-283bf66a9eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:20.099400 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c65c9b-d720-4cb6-9e3b-283bf66a9eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:20.099400 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c65c9b-d720-4cb6-9e3b-283bf66a9eb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:20.105152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c65c9b-d720-4cb6-9e3b-283bf66a9eb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:20.105152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c65c9b-d720-4cb6-9e3b-283bf66a9eb9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:20.114548 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1253/2521] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:53:20.199987 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1269/2522] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:53:20 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:53:20.202080 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d456d-0d84-45ab-8488-e5c93478ecd9 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:20.212169 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d456d-0d84-45ab-8488-e5c93478ecd9 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:20.217106 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d4254-e0a0-4ad3-8b70-baf189aca809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:20.217780 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d4254-e0a0-4ad3-8b70-baf189aca809 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:20.218065 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d4254-e0a0-4ad3-8b70-baf189aca809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:20.218534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d4254-e0a0-4ad3-8b70-baf189aca809 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:20.219121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d4254-e0a0-4ad3-8b70-baf189aca809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:20.220796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d4254-e0a0-4ad3-8b70-baf189aca809 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:20.232810 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1254/2523] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:20.236948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c65c9b-d720-4cb6-9e3b-283bf66a9eb9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:20.257134 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c65c9b-d720-4cb6-9e3b-283bf66a9eb9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:20.262079 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bdc81-b17a-42b8-87e8-9760f69605f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:20.262912 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bdc81-b17a-42b8-87e8-9760f69605f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:20.263196 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bdc81-b17a-42b8-87e8-9760f69605f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:20.263476 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bdc81-b17a-42b8-87e8-9760f69605f2 admin admin] PATH_INFO: `/v3/projects/44e3dce0910241f5938eb41b227d4d7e/users/ed90ba493b684523b4b623a897fda0ed/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:20.266070 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:20.281580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5bdc81-b17a-42b8-87e8-9760f69605f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44e3dce0910241f5938eb41b227d4d7e, user_id=ed90ba493b684523b4b623a897fda0ed, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:20.284336 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5bdc81-b17a-42b8-87e8-9760f69605f2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:20.374972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1270/2524] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:53:20 2022] PUT /identity/v3/projects/44e3dce0910241f5938eb41b227d4d7e/users/ed90ba493b684523b4b623a897fda0ed/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:20.379285 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d4254-e0a0-4ad3-8b70-baf189aca809 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:20.390153 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d4254-e0a0-4ad3-8b70-baf189aca809 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:20.451526 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe16b0d6-a204-41b9-bfd2-8542fd2bf29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:20.452227 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16b0d6-a204-41b9-bfd2-8542fd2bf29c admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:20.452493 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16b0d6-a204-41b9-bfd2-8542fd2bf29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:20.452790 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16b0d6-a204-41b9-bfd2-8542fd2bf29c admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:20.453523 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe16b0d6-a204-41b9-bfd2-8542fd2bf29c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:20.455243 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe16b0d6-a204-41b9-bfd2-8542fd2bf29c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:20.467905 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1255/2525] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:20.476523 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bdc81-b17a-42b8-87e8-9760f69605f2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:20.487959 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bdc81-b17a-42b8-87e8-9760f69605f2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:20.493885 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99acae-a51a-4b75-babc-196f34286d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:20.494804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99acae-a51a-4b75-babc-196f34286d74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:20.495090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99acae-a51a-4b75-babc-196f34286d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:20.495374 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99acae-a51a-4b75-babc-196f34286d74 admin admin] PATH_INFO: `/v3/projects/44e3dce0910241f5938eb41b227d4d7e/users/ed90ba493b684523b4b623a897fda0ed/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:20.497683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99acae-a51a-4b75-babc-196f34286d74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44e3dce0910241f5938eb41b227d4d7e, user_id=ed90ba493b684523b4b623a897fda0ed, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:20.500517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99acae-a51a-4b75-babc-196f34286d74 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:20.511938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1271/2526] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:53:20 2022] PUT /identity/v3/projects/44e3dce0910241f5938eb41b227d4d7e/users/ed90ba493b684523b4b623a897fda0ed/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:53:20.529138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7c631-d433-464f-add0-2ccc08912487 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:20.529333 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7c631-d433-464f-add0-2ccc08912487 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:20.529640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7c631-d433-464f-add0-2ccc08912487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:20.559069 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-ccf7c631-d433-464f-add0-2ccc08912487 None None] MFA Rules not processed for user `ed90ba493b684523b4b623a897fda0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:20.560379 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ccf7c631-d433-464f-add0-2ccc08912487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:20.634980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1256/2527] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:20 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:20.647628 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99acae-a51a-4b75-babc-196f34286d74 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:20.713721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99acae-a51a-4b75-babc-196f34286d74 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:20.718123 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c76a6e-df89-4aeb-b501-461ced0e6409 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:20.718828 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c76a6e-df89-4aeb-b501-461ced0e6409 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:20.719159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c76a6e-df89-4aeb-b501-461ced0e6409 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:20.719394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c76a6e-df89-4aeb-b501-461ced0e6409 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:20.732678 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c76a6e-df89-4aeb-b501-461ced0e6409 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:20.734149 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c76a6e-df89-4aeb-b501-461ced0e6409 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:20.745981 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1272/2528] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:53:20.882493 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7c631-d433-464f-add0-2ccc08912487 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:20.892771 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7c631-d433-464f-add0-2ccc08912487 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:20.897694 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f214b-961c-4b33-a0d3-38c1d47ba221 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:20.898431 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f214b-961c-4b33-a0d3-38c1d47ba221 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:20.898721 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f214b-961c-4b33-a0d3-38c1d47ba221 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:20.899001 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f214b-961c-4b33-a0d3-38c1d47ba221 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:20.908731 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:20.972555 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f214b-961c-4b33-a0d3-38c1d47ba221 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:20.974041 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f214b-961c-4b33-a0d3-38c1d47ba221 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:20.985344 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1257/2529] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:53:21.644497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c76a6e-df89-4aeb-b501-461ced0e6409 service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:21.722422 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c76a6e-df89-4aeb-b501-461ced0e6409 service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:21.728296 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75f6da-7b77-40ec-bc0a-c95e1066cff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:21.729187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75f6da-7b77-40ec-bc0a-c95e1066cff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:21.729466 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75f6da-7b77-40ec-bc0a-c95e1066cff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:21.729746 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75f6da-7b77-40ec-bc0a-c95e1066cff1 admin admin] PATH_INFO: `/v3/users/ef46e5b7c97a4a8196476141700b0aef` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:21.731016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75f6da-7b77-40ec-bc0a-c95e1066cff1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef46e5b7c97a4a8196476141700b0aef)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:21.733013 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75f6da-7b77-40ec-bc0a-c95e1066cff1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:21.985666 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ce75f6da-7b77-40ec-bc0a-c95e1066cff1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef46e5b7c97a4a8196476141700b0aef'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:21.986464 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ce75f6da-7b77-40ec-bc0a-c95e1066cff1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef46e5b7c97a4a8196476141700b0aef'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:21.987088 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ce75f6da-7b77-40ec-bc0a-c95e1066cff1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef46e5b7c97a4a8196476141700b0aef'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:22.001320 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ce75f6da-7b77-40ec-bc0a-c95e1066cff1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef46e5b7c97a4a8196476141700b0aef'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:22.009839 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ce75f6da-7b77-40ec-bc0a-c95e1066cff1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef46e5b7c97a4a8196476141700b0aef'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:22.012748 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:22.157025 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1273/2530] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:53:21 2022] DELETE /identity/v3/users/ef46e5b7c97a4a8196476141700b0aef => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:22.171345 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d05f6-29a1-47de-a099-3891a4e2b5bd None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:22.171856 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d05f6-29a1-47de-a099-3891a4e2b5bd None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:22.171954 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d05f6-29a1-47de-a099-3891a4e2b5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:22.189767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a10d05f6-29a1-47de-a099-3891a4e2b5bd None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:22.191392 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a10d05f6-29a1-47de-a099-3891a4e2b5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:22.273198 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1258/2531] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:22.285623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75f6da-7b77-40ec-bc0a-c95e1066cff1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:22.288744 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ce75f6da-7b77-40ec-bc0a-c95e1066cff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:22.360956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75f6da-7b77-40ec-bc0a-c95e1066cff1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:22.367270 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40678245-3fae-4147-8e3d-0674f7068f35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:22.368066 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40678245-3fae-4147-8e3d-0674f7068f35 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:22.368420 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40678245-3fae-4147-8e3d-0674f7068f35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:22.368767 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40678245-3fae-4147-8e3d-0674f7068f35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:22.379801 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40678245-3fae-4147-8e3d-0674f7068f35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:22.382396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40678245-3fae-4147-8e3d-0674f7068f35 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:22.394797 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1274/2532] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:53:23.295078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d05f6-29a1-47de-a099-3891a4e2b5bd None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:23.298117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a10d05f6-29a1-47de-a099-3891a4e2b5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:23.327707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d05f6-29a1-47de-a099-3891a4e2b5bd None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:23.352680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a50cf-f365-4e8e-b557-42fdf5d00ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:23.353584 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a50cf-f365-4e8e-b557-42fdf5d00ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:23.353860 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a50cf-f365-4e8e-b557-42fdf5d00ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:23.354132 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a50cf-f365-4e8e-b557-42fdf5d00ded admin admin] PATH_INFO: `/v3/projects/93f244b23dab47ca99a30773d83ffd46` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:23.355436 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a50cf-f365-4e8e-b557-42fdf5d00ded admin admin] RBAC: Authorizing `identity:delete_project(project_id=93f244b23dab47ca99a30773d83ffd46)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:23.357440 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a50cf-f365-4e8e-b557-42fdf5d00ded admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:23.544656 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-732a50cf-f365-4e8e-b557-42fdf5d00ded admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93f244b23dab47ca99a30773d83ffd46', 'request_id': 'req-732a50cf-f365-4e8e-b557-42fdf5d00ded'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:23.555325 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-732a50cf-f365-4e8e-b557-42fdf5d00ded admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93f244b23dab47ca99a30773d83ffd46', 'request_id': 'req-732a50cf-f365-4e8e-b557-42fdf5d00ded'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:23.562920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-732a50cf-f365-4e8e-b557-42fdf5d00ded admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93f244b23dab47ca99a30773d83ffd46', 'request_id': 'req-732a50cf-f365-4e8e-b557-42fdf5d00ded'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:23.751738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-732a50cf-f365-4e8e-b557-42fdf5d00ded admin admin] The token cache is being invalidate because project 93f244b23dab47ca99a30773d83ffd46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:53:23.752035 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-732a50cf-f365-4e8e-b557-42fdf5d00ded admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:23.752792 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-732a50cf-f365-4e8e-b557-42fdf5d00ded admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:23.754408 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1259/2533] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:53:23 2022] DELETE /identity/v3/projects/93f244b23dab47ca99a30773d83ffd46 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:23.758978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40678245-3fae-4147-8e3d-0674f7068f35 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:23.769599 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40678245-3fae-4147-8e3d-0674f7068f35 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:23.879652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21d12e-7ce0-49c0-9997-5bc0c02e5930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:23.880595 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21d12e-7ce0-49c0-9997-5bc0c02e5930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:23.880909 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21d12e-7ce0-49c0-9997-5bc0c02e5930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:23.881217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21d12e-7ce0-49c0-9997-5bc0c02e5930 admin admin] PATH_INFO: `/v3/users/ed90ba493b684523b4b623a897fda0ed` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:23.884045 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21d12e-7ce0-49c0-9997-5bc0c02e5930 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed90ba493b684523b4b623a897fda0ed)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:23.886217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21d12e-7ce0-49c0-9997-5bc0c02e5930 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:24.161855 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4e21d12e-7ce0-49c0-9997-5bc0c02e5930 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed90ba493b684523b4b623a897fda0ed'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:24.163303 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4e21d12e-7ce0-49c0-9997-5bc0c02e5930 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed90ba493b684523b4b623a897fda0ed'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:24.164208 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4e21d12e-7ce0-49c0-9997-5bc0c02e5930 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed90ba493b684523b4b623a897fda0ed'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:24.178552 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4e21d12e-7ce0-49c0-9997-5bc0c02e5930 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed90ba493b684523b4b623a897fda0ed'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:24.186306 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-4e21d12e-7ce0-49c0-9997-5bc0c02e5930 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed90ba493b684523b4b623a897fda0ed'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:24.292972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1275/2534] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:53:23 2022] DELETE /identity/v3/users/ed90ba493b684523b4b623a897fda0ed => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:24.577249 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a50cf-f365-4e8e-b557-42fdf5d00ded admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:24.579856 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-732a50cf-f365-4e8e-b557-42fdf5d00ded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:24.602398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a50cf-f365-4e8e-b557-42fdf5d00ded admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:24.648272 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:24.679843 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc77273-5921-4b4d-9230-52410fed0321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:24.680732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc77273-5921-4b4d-9230-52410fed0321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:24.681018 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc77273-5921-4b4d-9230-52410fed0321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:24.681294 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc77273-5921-4b4d-9230-52410fed0321 admin admin] PATH_INFO: `/v3/users/4e6b788e0819483580c09902a341ea6d` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:24.682668 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc77273-5921-4b4d-9230-52410fed0321 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e6b788e0819483580c09902a341ea6d)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:24.684715 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc77273-5921-4b4d-9230-52410fed0321 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:24.899458 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21d12e-7ce0-49c0-9997-5bc0c02e5930 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:24.900815 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4e21d12e-7ce0-49c0-9997-5bc0c02e5930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:24.916725 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21d12e-7ce0-49c0-9997-5bc0c02e5930 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:24.922137 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc007b1-8060-4a7c-9150-7f883682ce62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:24.926878 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc007b1-8060-4a7c-9150-7f883682ce62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:24.926878 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc007b1-8060-4a7c-9150-7f883682ce62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:24.926878 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc007b1-8060-4a7c-9150-7f883682ce62 admin admin] PATH_INFO: `/v3/projects/44e3dce0910241f5938eb41b227d4d7e` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:24.926878 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc007b1-8060-4a7c-9150-7f883682ce62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44e3dce0910241f5938eb41b227d4d7e)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:24.927983 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc007b1-8060-4a7c-9150-7f883682ce62 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:24.936445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:24.961741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ddc77273-5921-4b4d-9230-52410fed0321 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e6b788e0819483580c09902a341ea6d'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:24.962512 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ddc77273-5921-4b4d-9230-52410fed0321 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e6b788e0819483580c09902a341ea6d'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:24.963120 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ddc77273-5921-4b4d-9230-52410fed0321 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e6b788e0819483580c09902a341ea6d'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:24.976140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ddc77273-5921-4b4d-9230-52410fed0321 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e6b788e0819483580c09902a341ea6d'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:24.986288 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-ddc77273-5921-4b4d-9230-52410fed0321 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e6b788e0819483580c09902a341ea6d'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:25.056576 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2fc007b1-8060-4a7c-9150-7f883682ce62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44e3dce0910241f5938eb41b227d4d7e', 'request_id': 'req-2fc007b1-8060-4a7c-9150-7f883682ce62'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:25.062295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2fc007b1-8060-4a7c-9150-7f883682ce62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44e3dce0910241f5938eb41b227d4d7e', 'request_id': 'req-2fc007b1-8060-4a7c-9150-7f883682ce62'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:25.070397 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2fc007b1-8060-4a7c-9150-7f883682ce62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44e3dce0910241f5938eb41b227d4d7e', 'request_id': 'req-2fc007b1-8060-4a7c-9150-7f883682ce62'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:25.075086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1260/2535] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:53:24 2022] DELETE /identity/v3/users/4e6b788e0819483580c09902a341ea6d => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:25.241442 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2fc007b1-8060-4a7c-9150-7f883682ce62 admin admin] The token cache is being invalidate because project 44e3dce0910241f5938eb41b227d4d7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:53:25.241750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2fc007b1-8060-4a7c-9150-7f883682ce62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:25.242817 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2fc007b1-8060-4a7c-9150-7f883682ce62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:25.244472 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1276/2536] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:53:24 2022] DELETE /identity/v3/projects/44e3dce0910241f5938eb41b227d4d7e => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:53:25.343300 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc77273-5921-4b4d-9230-52410fed0321 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:25.346849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ddc77273-5921-4b4d-9230-52410fed0321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:25.467453 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc77273-5921-4b4d-9230-52410fed0321 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:25.472875 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af03e40-4f83-40a5-831f-945d065ba375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:25.475611 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af03e40-4f83-40a5-831f-945d065ba375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:25.476140 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af03e40-4f83-40a5-831f-945d065ba375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:25.476628 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af03e40-4f83-40a5-831f-945d065ba375 admin admin] PATH_INFO: `/v3/projects/ab95af0a9cb74221a15ac73cbb8ca0ff` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:25.478180 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af03e40-4f83-40a5-831f-945d065ba375 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab95af0a9cb74221a15ac73cbb8ca0ff)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:25.480394 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af03e40-4f83-40a5-831f-945d065ba375 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:25.621893 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8af03e40-4f83-40a5-831f-945d065ba375 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab95af0a9cb74221a15ac73cbb8ca0ff', 'request_id': 'req-8af03e40-4f83-40a5-831f-945d065ba375'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:25.625787 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:25.627732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8af03e40-4f83-40a5-831f-945d065ba375 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab95af0a9cb74221a15ac73cbb8ca0ff', 'request_id': 'req-8af03e40-4f83-40a5-831f-945d065ba375'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:25.634716 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8af03e40-4f83-40a5-831f-945d065ba375 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab95af0a9cb74221a15ac73cbb8ca0ff', 'request_id': 'req-8af03e40-4f83-40a5-831f-945d065ba375'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:25.803466 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8af03e40-4f83-40a5-831f-945d065ba375 admin admin] The token cache is being invalidate because project ab95af0a9cb74221a15ac73cbb8ca0ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:53:25.807792 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8af03e40-4f83-40a5-831f-945d065ba375 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:25.807792 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-8af03e40-4f83-40a5-831f-945d065ba375 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:25.807792 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1261/2537] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:53:25 2022] DELETE /identity/v3/projects/ab95af0a9cb74221a15ac73cbb8ca0ff => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:25.812561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba7954-bf6c-4c65-91f3-e7bc3688adaa None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:25.812933 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba7954-bf6c-4c65-91f3-e7bc3688adaa None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:25.813741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba7954-bf6c-4c65-91f3-e7bc3688adaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:25.843112 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-69ba7954-bf6c-4c65-91f3-e7bc3688adaa None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:25.845380 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-69ba7954-bf6c-4c65-91f3-e7bc3688adaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:25.925625 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1277/2538] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:25.935221 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc631e-6e8d-4c82-9cf2-cd80e673e747 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:25.935549 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc631e-6e8d-4c82-9cf2-cd80e673e747 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:25.935863 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc631e-6e8d-4c82-9cf2-cd80e673e747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:25.956678 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-74bc631e-6e8d-4c82-9cf2-cd80e673e747 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:25.958604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-74bc631e-6e8d-4c82-9cf2-cd80e673e747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:25.969469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1262/2539] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:25.973792 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba7954-bf6c-4c65-91f3-e7bc3688adaa None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:25.984059 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba7954-bf6c-4c65-91f3-e7bc3688adaa None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:25.988857 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc580a-8c6e-4e52-a765-cf4e5aed6bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:25.989524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc580a-8c6e-4e52-a765-cf4e5aed6bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:25.989815 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc580a-8c6e-4e52-a765-cf4e5aed6bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:25.990101 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc580a-8c6e-4e52-a765-cf4e5aed6bd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:25.991382 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc580a-8c6e-4e52-a765-cf4e5aed6bd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:53:25.991709 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc580a-8c6e-4e52-a765-cf4e5aed6bd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:25.993573 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc580a-8c6e-4e52-a765-cf4e5aed6bd0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:26.008295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1278/2540] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:53:26.013339 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc631e-6e8d-4c82-9cf2-cd80e673e747 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:26.022999 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc631e-6e8d-4c82-9cf2-cd80e673e747 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:26.027846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd64a60-09fd-45a5-9f9e-22fdf5396e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:26.028658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd64a60-09fd-45a5-9f9e-22fdf5396e68 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:26.028949 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd64a60-09fd-45a5-9f9e-22fdf5396e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:26.029255 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd64a60-09fd-45a5-9f9e-22fdf5396e68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:26.029951 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd64a60-09fd-45a5-9f9e-22fdf5396e68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:26.031917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd64a60-09fd-45a5-9f9e-22fdf5396e68 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:26.157940 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1263/2541] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:53:26 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:26.161601 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc580a-8c6e-4e52-a765-cf4e5aed6bd0 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:26.172689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc580a-8c6e-4e52-a765-cf4e5aed6bd0 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:26.240889 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152ef19-0130-49cc-b85a-0af5a38bfb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:26.241646 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152ef19-0130-49cc-b85a-0af5a38bfb5d admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:26.241923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152ef19-0130-49cc-b85a-0af5a38bfb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:26.242197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152ef19-0130-49cc-b85a-0af5a38bfb5d admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:26.242901 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0152ef19-0130-49cc-b85a-0af5a38bfb5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:26.244875 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0152ef19-0130-49cc-b85a-0af5a38bfb5d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:26.331754 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1279/2542] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:53:26 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:53:26.336309 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd64a60-09fd-45a5-9f9e-22fdf5396e68 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:26.347810 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd64a60-09fd-45a5-9f9e-22fdf5396e68 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:26.353696 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd70c1f-119d-40d7-9f71-b2ffa0ab15c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:26.354385 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd70c1f-119d-40d7-9f71-b2ffa0ab15c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:26.354672 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd70c1f-119d-40d7-9f71-b2ffa0ab15c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:26.354943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd70c1f-119d-40d7-9f71-b2ffa0ab15c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:26.355822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd70c1f-119d-40d7-9f71-b2ffa0ab15c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:26.357365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd70c1f-119d-40d7-9f71-b2ffa0ab15c2 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:26.369845 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1264/2543] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:26.374394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152ef19-0130-49cc-b85a-0af5a38bfb5d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:26.386196 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152ef19-0130-49cc-b85a-0af5a38bfb5d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:26.391434 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be96e4-26ab-4df4-b916-58e96852b493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:26.392179 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be96e4-26ab-4df4-b916-58e96852b493 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:26.392464 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be96e4-26ab-4df4-b916-58e96852b493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:26.392741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be96e4-26ab-4df4-b916-58e96852b493 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:26.393481 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be96e4-26ab-4df4-b916-58e96852b493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:26.395210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be96e4-26ab-4df4-b916-58e96852b493 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:26.406305 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1280/2544] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:26.410260 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd70c1f-119d-40d7-9f71-b2ffa0ab15c2 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:26.422374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd70c1f-119d-40d7-9f71-b2ffa0ab15c2 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:26.426984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b916aa8-0ca7-463d-bea7-5b71d8542c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:26.428720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b916aa8-0ca7-463d-bea7-5b71d8542c71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:26.428720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b916aa8-0ca7-463d-bea7-5b71d8542c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:26.428720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b916aa8-0ca7-463d-bea7-5b71d8542c71 admin admin] PATH_INFO: `/v3/projects/9cf31b10285b46d692eda809c89d817c/users/e3489a4bf9bf42b884a923bcafbef7b0/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:26.445959 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b916aa8-0ca7-463d-bea7-5b71d8542c71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cf31b10285b46d692eda809c89d817c, user_id=e3489a4bf9bf42b884a923bcafbef7b0, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:26.448805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b916aa8-0ca7-463d-bea7-5b71d8542c71 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:26.522051 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1265/2545] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:53:26 2022] PUT /identity/v3/projects/9cf31b10285b46d692eda809c89d817c/users/e3489a4bf9bf42b884a923bcafbef7b0/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:26.532991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5324414-3a40-4b65-a530-c5d089dd40ed None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:26.533471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5324414-3a40-4b65-a530-c5d089dd40ed None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:26.533920 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5324414-3a40-4b65-a530-c5d089dd40ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:26.541278 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:26.564047 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d5324414-3a40-4b65-a530-c5d089dd40ed None None] MFA Rules not processed for user `e3489a4bf9bf42b884a923bcafbef7b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:26.565264 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d5324414-3a40-4b65-a530-c5d089dd40ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:26.635133 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1281/2546] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:26.647166 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b916aa8-0ca7-463d-bea7-5b71d8542c71 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:26.649110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8b916aa8-0ca7-463d-bea7-5b71d8542c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:26.713309 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b916aa8-0ca7-463d-bea7-5b71d8542c71 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:26.717204 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72ac91-a5e1-4fe5-9e23-5333f05afceb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:26.717868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72ac91-a5e1-4fe5-9e23-5333f05afceb service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:26.718148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72ac91-a5e1-4fe5-9e23-5333f05afceb service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:26.718418 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72ac91-a5e1-4fe5-9e23-5333f05afceb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:26.729002 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72ac91-a5e1-4fe5-9e23-5333f05afceb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:26.730456 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72ac91-a5e1-4fe5-9e23-5333f05afceb service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:26.740216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1266/2547] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:53:26.762249 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5324414-3a40-4b65-a530-c5d089dd40ed None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:26.829992 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5324414-3a40-4b65-a530-c5d089dd40ed None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:26.835383 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f9053-29aa-43a8-9659-95e13a7cb718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:26.836322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f9053-29aa-43a8-9659-95e13a7cb718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:26.836605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f9053-29aa-43a8-9659-95e13a7cb718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:26.836885 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f9053-29aa-43a8-9659-95e13a7cb718 admin admin] PATH_INFO: `/v3/users/e3489a4bf9bf42b884a923bcafbef7b0` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:26.838117 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f9053-29aa-43a8-9659-95e13a7cb718 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3489a4bf9bf42b884a923bcafbef7b0)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:26.840138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f9053-29aa-43a8-9659-95e13a7cb718 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:27.097625 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bd6f9053-29aa-43a8-9659-95e13a7cb718 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3489a4bf9bf42b884a923bcafbef7b0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:27.098277 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bd6f9053-29aa-43a8-9659-95e13a7cb718 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3489a4bf9bf42b884a923bcafbef7b0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:27.098867 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bd6f9053-29aa-43a8-9659-95e13a7cb718 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3489a4bf9bf42b884a923bcafbef7b0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:27.111269 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bd6f9053-29aa-43a8-9659-95e13a7cb718 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3489a4bf9bf42b884a923bcafbef7b0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:27.118035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-bd6f9053-29aa-43a8-9659-95e13a7cb718 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3489a4bf9bf42b884a923bcafbef7b0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:27.237698 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1282/2548] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:53:26 2022] DELETE /identity/v3/users/e3489a4bf9bf42b884a923bcafbef7b0 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:27.242519 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b16f8-7173-4c23-88f2-44f46bb99a26 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:27.242818 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b16f8-7173-4c23-88f2-44f46bb99a26 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:27.243096 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b16f8-7173-4c23-88f2-44f46bb99a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:27.264921 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d58b16f8-7173-4c23-88f2-44f46bb99a26 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:27.264921 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d58b16f8-7173-4c23-88f2-44f46bb99a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:27.323860 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1267/2549] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:27.335473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f9053-29aa-43a8-9659-95e13a7cb718 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:27.337580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bd6f9053-29aa-43a8-9659-95e13a7cb718 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:27.351726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:27.399198 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f9053-29aa-43a8-9659-95e13a7cb718 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:27.403022 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944ec09-cdaa-48db-a8f7-a9bdb60eecac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:27.403685 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944ec09-cdaa-48db-a8f7-a9bdb60eecac service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:27.403984 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944ec09-cdaa-48db-a8f7-a9bdb60eecac service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:27.404259 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944ec09-cdaa-48db-a8f7-a9bdb60eecac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:27.414561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944ec09-cdaa-48db-a8f7-a9bdb60eecac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:27.416031 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944ec09-cdaa-48db-a8f7-a9bdb60eecac service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:27.425856 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1283/2550] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:53:27.865497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b16f8-7173-4c23-88f2-44f46bb99a26 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:27.867634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d58b16f8-7173-4c23-88f2-44f46bb99a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:27.890527 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b16f8-7173-4c23-88f2-44f46bb99a26 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:27.890527 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098f3b2-d181-49bb-b968-0764f2c570dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:27.890527 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098f3b2-d181-49bb-b968-0764f2c570dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:27.890527 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098f3b2-d181-49bb-b968-0764f2c570dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:27.891458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098f3b2-d181-49bb-b968-0764f2c570dd admin admin] PATH_INFO: `/v3/projects/9cf31b10285b46d692eda809c89d817c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:27.891458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098f3b2-d181-49bb-b968-0764f2c570dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cf31b10285b46d692eda809c89d817c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:27.891458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098f3b2-d181-49bb-b968-0764f2c570dd admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:27.909519 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:28.008722 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3098f3b2-d181-49bb-b968-0764f2c570dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cf31b10285b46d692eda809c89d817c', 'request_id': 'req-3098f3b2-d181-49bb-b968-0764f2c570dd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:28.014482 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3098f3b2-d181-49bb-b968-0764f2c570dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cf31b10285b46d692eda809c89d817c', 'request_id': 'req-3098f3b2-d181-49bb-b968-0764f2c570dd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:28.021080 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3098f3b2-d181-49bb-b968-0764f2c570dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cf31b10285b46d692eda809c89d817c', 'request_id': 'req-3098f3b2-d181-49bb-b968-0764f2c570dd'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:28.118416 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3098f3b2-d181-49bb-b968-0764f2c570dd admin admin] The token cache is being invalidate because project 9cf31b10285b46d692eda809c89d817c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:53:28.118625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3098f3b2-d181-49bb-b968-0764f2c570dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:28.119362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3098f3b2-d181-49bb-b968-0764f2c570dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:28.124250 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1268/2551] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:53:27 2022] DELETE /identity/v3/projects/9cf31b10285b46d692eda809c89d817c => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:28.126609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42dfcf-77c3-48ed-8940-6d3d2a7340dc None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:28.127174 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42dfcf-77c3-48ed-8940-6d3d2a7340dc None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:28.127271 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42dfcf-77c3-48ed-8940-6d3d2a7340dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:28.146273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ab42dfcf-77c3-48ed-8940-6d3d2a7340dc None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:28.147954 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ab42dfcf-77c3-48ed-8940-6d3d2a7340dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:28.230110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1284/2552] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:28.237908 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49350be4-ee19-493d-a870-b6877548df15 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:28.238333 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49350be4-ee19-493d-a870-b6877548df15 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:28.238742 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49350be4-ee19-493d-a870-b6877548df15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:28.267341 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-49350be4-ee19-493d-a870-b6877548df15 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:28.269903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-49350be4-ee19-493d-a870-b6877548df15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:28.283894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1269/2553] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:53:28.294100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42dfcf-77c3-48ed-8940-6d3d2a7340dc None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:28.306220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42dfcf-77c3-48ed-8940-6d3d2a7340dc None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:28.311161 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74164da7-7060-4fca-ae3e-97bd5f3a208d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:28.311871 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74164da7-7060-4fca-ae3e-97bd5f3a208d admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:28.312151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74164da7-7060-4fca-ae3e-97bd5f3a208d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:28.312447 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74164da7-7060-4fca-ae3e-97bd5f3a208d admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:28.313198 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74164da7-7060-4fca-ae3e-97bd5f3a208d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:53:28.313623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74164da7-7060-4fca-ae3e-97bd5f3a208d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:28.315141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74164da7-7060-4fca-ae3e-97bd5f3a208d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:28.328777 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1285/2554] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:53:28.333907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49350be4-ee19-493d-a870-b6877548df15 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:28.344817 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49350be4-ee19-493d-a870-b6877548df15 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:28.349600 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7cf69-c3cb-48c8-ab5a-fce2d03dcd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:28.350259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7cf69-c3cb-48c8-ab5a-fce2d03dcd53 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:28.351295 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7cf69-c3cb-48c8-ab5a-fce2d03dcd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:28.351648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7cf69-c3cb-48c8-ab5a-fce2d03dcd53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:28.352457 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7cf69-c3cb-48c8-ab5a-fce2d03dcd53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:28.354374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7cf69-c3cb-48c8-ab5a-fce2d03dcd53 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:28.456574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1270/2555] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:53:28 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:28.461337 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74164da7-7060-4fca-ae3e-97bd5f3a208d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:28.471608 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74164da7-7060-4fca-ae3e-97bd5f3a208d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:28.530493 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb40c1c-04a7-4a77-a744-ec082bbbaa1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:28.531184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb40c1c-04a7-4a77-a744-ec082bbbaa1b admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:28.531501 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb40c1c-04a7-4a77-a744-ec082bbbaa1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:28.531817 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb40c1c-04a7-4a77-a744-ec082bbbaa1b admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:28.532539 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb40c1c-04a7-4a77-a744-ec082bbbaa1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:28.534537 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb40c1c-04a7-4a77-a744-ec082bbbaa1b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:28.644010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1286/2556] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:53:28 2022] POST /identity/v3/users => generated 287 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:53:28.645525 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7cf69-c3cb-48c8-ab5a-fce2d03dcd53 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:28.657249 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7cf69-c3cb-48c8-ab5a-fce2d03dcd53 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:28.661962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06af89-c934-4a73-a1c4-ac9b6e52de91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:28.662560 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06af89-c934-4a73-a1c4-ac9b6e52de91 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:28.662844 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06af89-c934-4a73-a1c4-ac9b6e52de91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:28.663113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06af89-c934-4a73-a1c4-ac9b6e52de91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:28.663872 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06af89-c934-4a73-a1c4-ac9b6e52de91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:28.665452 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06af89-c934-4a73-a1c4-ac9b6e52de91 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:28.676419 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1271/2557] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:28.681385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb40c1c-04a7-4a77-a744-ec082bbbaa1b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:28.692734 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb40c1c-04a7-4a77-a744-ec082bbbaa1b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:28.698475 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227504f4-3d89-48af-ba23-58559b6d2c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:28.699138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227504f4-3d89-48af-ba23-58559b6d2c27 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:28.699427 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227504f4-3d89-48af-ba23-58559b6d2c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:28.699739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227504f4-3d89-48af-ba23-58559b6d2c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:28.700601 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227504f4-3d89-48af-ba23-58559b6d2c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:28.702507 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227504f4-3d89-48af-ba23-58559b6d2c27 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:28.713367 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1287/2558] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:28.717676 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06af89-c934-4a73-a1c4-ac9b6e52de91 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:28.729095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06af89-c934-4a73-a1c4-ac9b6e52de91 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:28.733854 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5242b-bc96-477b-9953-c8e457b35f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:28.734738 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5242b-bc96-477b-9953-c8e457b35f0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:28.735028 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5242b-bc96-477b-9953-c8e457b35f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:28.735336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5242b-bc96-477b-9953-c8e457b35f0d admin admin] PATH_INFO: `/v3/projects/d36b9243f83546ac8c77dd177f4f63cf/users/07573e64a5264af2bb5060cbab91d38f/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:28.750953 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5242b-bc96-477b-9953-c8e457b35f0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d36b9243f83546ac8c77dd177f4f63cf, user_id=07573e64a5264af2bb5060cbab91d38f, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:28.753675 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5242b-bc96-477b-9953-c8e457b35f0d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:28.827059 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1272/2559] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:53:28 2022] PUT /identity/v3/projects/d36b9243f83546ac8c77dd177f4f63cf/users/07573e64a5264af2bb5060cbab91d38f/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:28.839412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76850c43-fe8e-41e5-ab75-3883d535c9b9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:28.839737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76850c43-fe8e-41e5-ab75-3883d535c9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:28.840016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76850c43-fe8e-41e5-ab75-3883d535c9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:28.867519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-76850c43-fe8e-41e5-ab75-3883d535c9b9 None None] MFA Rules not processed for user `07573e64a5264af2bb5060cbab91d38f`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:28.868995 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-76850c43-fe8e-41e5-ab75-3883d535c9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:28.877645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:28.937217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1288/2560] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:28.942102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5242b-bc96-477b-9953-c8e457b35f0d admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:29.014593 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5242b-bc96-477b-9953-c8e457b35f0d admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:29.019413 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092ce1b-1ca7-4bda-bef8-9d18e0c67d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:29.020107 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092ce1b-1ca7-4bda-bef8-9d18e0c67d72 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:29.020392 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092ce1b-1ca7-4bda-bef8-9d18e0c67d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:29.020665 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092ce1b-1ca7-4bda-bef8-9d18e0c67d72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:29.021357 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092ce1b-1ca7-4bda-bef8-9d18e0c67d72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:29.023388 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092ce1b-1ca7-4bda-bef8-9d18e0c67d72 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:29.153747 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1273/2561] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:53:28 2022] POST /identity/v3/projects => generated 375 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:29.158187 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76850c43-fe8e-41e5-ab75-3883d535c9b9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:29.170205 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76850c43-fe8e-41e5-ab75-3883d535c9b9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:29.236565 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094550d1-1f44-4aa7-9d48-3d8b3912ac2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:29.237329 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094550d1-1f44-4aa7-9d48-3d8b3912ac2c admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:29.237689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094550d1-1f44-4aa7-9d48-3d8b3912ac2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:29.238045 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094550d1-1f44-4aa7-9d48-3d8b3912ac2c admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:29.238924 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094550d1-1f44-4aa7-9d48-3d8b3912ac2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:29.240865 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094550d1-1f44-4aa7-9d48-3d8b3912ac2c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:29.320178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1289/2562] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:53:29 2022] POST /identity/v3/users => generated 294 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:29.324232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092ce1b-1ca7-4bda-bef8-9d18e0c67d72 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:29.336262 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092ce1b-1ca7-4bda-bef8-9d18e0c67d72 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:29.341380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3328700-9a8b-4854-a541-cf291e0b8442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:29.342052 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3328700-9a8b-4854-a541-cf291e0b8442 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:29.342337 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3328700-9a8b-4854-a541-cf291e0b8442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:29.342613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3328700-9a8b-4854-a541-cf291e0b8442 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:29.343353 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3328700-9a8b-4854-a541-cf291e0b8442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:29.345068 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3328700-9a8b-4854-a541-cf291e0b8442 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:29.358961 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1274/2563] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:53:29.363284 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094550d1-1f44-4aa7-9d48-3d8b3912ac2c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:29.375629 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094550d1-1f44-4aa7-9d48-3d8b3912ac2c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:29.380711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3184e2-8c47-4691-8895-b8d68bd09c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:29.381679 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3184e2-8c47-4691-8895-b8d68bd09c97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:29.382636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3184e2-8c47-4691-8895-b8d68bd09c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:29.382950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3184e2-8c47-4691-8895-b8d68bd09c97 admin admin] PATH_INFO: `/v3/projects/c632a533a9004e48b9dd051d86d4e763/users/8d9d9a125f0d4c4b8e6875104fe228b2/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:29.407266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3184e2-8c47-4691-8895-b8d68bd09c97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c632a533a9004e48b9dd051d86d4e763, user_id=8d9d9a125f0d4c4b8e6875104fe228b2, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:29.410938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3184e2-8c47-4691-8895-b8d68bd09c97 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:29.746613 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1290/2564] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:53:29 2022] PUT /identity/v3/projects/c632a533a9004e48b9dd051d86d4e763/users/8d9d9a125f0d4c4b8e6875104fe228b2/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:29.750680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3328700-9a8b-4854-a541-cf291e0b8442 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:29.760839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3328700-9a8b-4854-a541-cf291e0b8442 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:29.773184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:29.829589 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e4571-7690-4ad2-a81a-942a62878f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:29.830357 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e4571-7690-4ad2-a81a-942a62878f19 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:29.830648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e4571-7690-4ad2-a81a-942a62878f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:29.830929 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e4571-7690-4ad2-a81a-942a62878f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:29.831678 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e4571-7690-4ad2-a81a-942a62878f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:29.837422 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e4571-7690-4ad2-a81a-942a62878f19 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:29.845193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1275/2565] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:29.849448 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3184e2-8c47-4691-8895-b8d68bd09c97 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:29.861919 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3184e2-8c47-4691-8895-b8d68bd09c97 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:29.867159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376dc669-07dd-4026-8db5-b6b48a8094f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:29.868103 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376dc669-07dd-4026-8db5-b6b48a8094f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:29.868387 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376dc669-07dd-4026-8db5-b6b48a8094f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:29.868662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376dc669-07dd-4026-8db5-b6b48a8094f7 admin admin] PATH_INFO: `/v3/projects/c632a533a9004e48b9dd051d86d4e763/users/8d9d9a125f0d4c4b8e6875104fe228b2/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:29.870650 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376dc669-07dd-4026-8db5-b6b48a8094f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c632a533a9004e48b9dd051d86d4e763, user_id=8d9d9a125f0d4c4b8e6875104fe228b2, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:29.873479 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376dc669-07dd-4026-8db5-b6b48a8094f7 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:29.883984 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1291/2566] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:53:29 2022] PUT /identity/v3/projects/c632a533a9004e48b9dd051d86d4e763/users/8d9d9a125f0d4c4b8e6875104fe228b2/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:53:29.894332 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be58eb-2348-42ff-a49a-4f7c0a4efd20 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:29.894626 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be58eb-2348-42ff-a49a-4f7c0a4efd20 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:29.894911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be58eb-2348-42ff-a49a-4f7c0a4efd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:29.925092 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d6be58eb-2348-42ff-a49a-4f7c0a4efd20 None None] MFA Rules not processed for user `8d9d9a125f0d4c4b8e6875104fe228b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:29.926315 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d6be58eb-2348-42ff-a49a-4f7c0a4efd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:30.002001 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1276/2567] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:30.013999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376dc669-07dd-4026-8db5-b6b48a8094f7 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:30.015953 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-376dc669-07dd-4026-8db5-b6b48a8094f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:30.077360 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376dc669-07dd-4026-8db5-b6b48a8094f7 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:30.081173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc686f0a-40bc-4238-b3fc-8db9bc7e5076 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:30.081829 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc686f0a-40bc-4238-b3fc-8db9bc7e5076 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:30.082111 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc686f0a-40bc-4238-b3fc-8db9bc7e5076 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:30.082423 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc686f0a-40bc-4238-b3fc-8db9bc7e5076 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:30.097414 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:30.142349 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc686f0a-40bc-4238-b3fc-8db9bc7e5076 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:30.143831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc686f0a-40bc-4238-b3fc-8db9bc7e5076 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:30.153854 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1292/2568] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:53:33.395452 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be58eb-2348-42ff-a49a-4f7c0a4efd20 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:33.468492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be58eb-2348-42ff-a49a-4f7c0a4efd20 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:33.474661 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83086173-7e97-4736-aa0d-a38d80b841b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:33.475585 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83086173-7e97-4736-aa0d-a38d80b841b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:33.475904 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83086173-7e97-4736-aa0d-a38d80b841b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:33.476198 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83086173-7e97-4736-aa0d-a38d80b841b6 admin admin] PATH_INFO: `/v3/users/07573e64a5264af2bb5060cbab91d38f` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:33.480014 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83086173-7e97-4736-aa0d-a38d80b841b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07573e64a5264af2bb5060cbab91d38f)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:33.480014 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83086173-7e97-4736-aa0d-a38d80b841b6 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:33.704473 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:33.712208 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-83086173-7e97-4736-aa0d-a38d80b841b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07573e64a5264af2bb5060cbab91d38f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:33.718089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-83086173-7e97-4736-aa0d-a38d80b841b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07573e64a5264af2bb5060cbab91d38f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:33.718089 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-83086173-7e97-4736-aa0d-a38d80b841b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07573e64a5264af2bb5060cbab91d38f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:33.727557 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-83086173-7e97-4736-aa0d-a38d80b841b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07573e64a5264af2bb5060cbab91d38f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:33.736026 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-83086173-7e97-4736-aa0d-a38d80b841b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07573e64a5264af2bb5060cbab91d38f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:33.826678 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1277/2569] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:53:33 2022] DELETE /identity/v3/users/07573e64a5264af2bb5060cbab91d38f => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:33.831419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38787ca5-0b3e-406e-b470-d648d1bc0a3e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:33.832059 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38787ca5-0b3e-406e-b470-d648d1bc0a3e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:33.832199 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38787ca5-0b3e-406e-b470-d648d1bc0a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:33.854033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-38787ca5-0b3e-406e-b470-d648d1bc0a3e None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:33.855282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-38787ca5-0b3e-406e-b470-d648d1bc0a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:33.936370 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1293/2570] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:33.946330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83086173-7e97-4736-aa0d-a38d80b841b6 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:33.947786 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-83086173-7e97-4736-aa0d-a38d80b841b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:34.028535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83086173-7e97-4736-aa0d-a38d80b841b6 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:34.032197 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b69880-7cbd-4f25-bc36-d6ef9e05c417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:34.032844 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b69880-7cbd-4f25-bc36-d6ef9e05c417 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:34.033131 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b69880-7cbd-4f25-bc36-d6ef9e05c417 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:34.033416 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b69880-7cbd-4f25-bc36-d6ef9e05c417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:34.043139 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b69880-7cbd-4f25-bc36-d6ef9e05c417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:34.044602 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b69880-7cbd-4f25-bc36-d6ef9e05c417 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:34.055101 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1278/2571] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:53:34.525598 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38787ca5-0b3e-406e-b470-d648d1bc0a3e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:34.527138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-38787ca5-0b3e-406e-b470-d648d1bc0a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:34.540963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38787ca5-0b3e-406e-b470-d648d1bc0a3e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:34.547627 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f8c4c-d2b0-47b2-a5db-71a5df6ec759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:34.548845 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f8c4c-d2b0-47b2-a5db-71a5df6ec759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:34.549566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f8c4c-d2b0-47b2-a5db-71a5df6ec759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:34.549566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f8c4c-d2b0-47b2-a5db-71a5df6ec759 admin admin] PATH_INFO: `/v3/projects/d36b9243f83546ac8c77dd177f4f63cf` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:34.550924 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f8c4c-d2b0-47b2-a5db-71a5df6ec759 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d36b9243f83546ac8c77dd177f4f63cf)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:34.552975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f8c4c-d2b0-47b2-a5db-71a5df6ec759 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:34.668770 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a90f8c4c-d2b0-47b2-a5db-71a5df6ec759 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd36b9243f83546ac8c77dd177f4f63cf', 'request_id': 'req-a90f8c4c-d2b0-47b2-a5db-71a5df6ec759'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:34.674388 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a90f8c4c-d2b0-47b2-a5db-71a5df6ec759 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd36b9243f83546ac8c77dd177f4f63cf', 'request_id': 'req-a90f8c4c-d2b0-47b2-a5db-71a5df6ec759'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:34.680705 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a90f8c4c-d2b0-47b2-a5db-71a5df6ec759 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd36b9243f83546ac8c77dd177f4f63cf', 'request_id': 'req-a90f8c4c-d2b0-47b2-a5db-71a5df6ec759'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:34.778675 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a90f8c4c-d2b0-47b2-a5db-71a5df6ec759 admin admin] The token cache is being invalidate because project d36b9243f83546ac8c77dd177f4f63cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:53:34.778896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a90f8c4c-d2b0-47b2-a5db-71a5df6ec759 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:34.779620 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a90f8c4c-d2b0-47b2-a5db-71a5df6ec759 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:34.782980 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1294/2572] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:53:34 2022] DELETE /identity/v3/projects/d36b9243f83546ac8c77dd177f4f63cf => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:34.785754 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b69880-7cbd-4f25-bc36-d6ef9e05c417 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:34.795810 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b69880-7cbd-4f25-bc36-d6ef9e05c417 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:34.856290 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32baf38-9899-4656-9052-f7b32e5467ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:34.857166 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32baf38-9899-4656-9052-f7b32e5467ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:34.857447 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32baf38-9899-4656-9052-f7b32e5467ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:34.857720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32baf38-9899-4656-9052-f7b32e5467ee admin admin] PATH_INFO: `/v3/users/8d9d9a125f0d4c4b8e6875104fe228b2` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:34.859083 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32baf38-9899-4656-9052-f7b32e5467ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d9d9a125f0d4c4b8e6875104fe228b2)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:34.861104 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32baf38-9899-4656-9052-f7b32e5467ee admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:35.099232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f32baf38-9899-4656-9052-f7b32e5467ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d9d9a125f0d4c4b8e6875104fe228b2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:35.100013 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f32baf38-9899-4656-9052-f7b32e5467ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d9d9a125f0d4c4b8e6875104fe228b2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:35.105341 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f32baf38-9899-4656-9052-f7b32e5467ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d9d9a125f0d4c4b8e6875104fe228b2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:35.111044 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:35.113514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f32baf38-9899-4656-9052-f7b32e5467ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d9d9a125f0d4c4b8e6875104fe228b2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:35.121506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f32baf38-9899-4656-9052-f7b32e5467ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d9d9a125f0d4c4b8e6875104fe228b2'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:35.211974 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1279/2573] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:53:34 2022] DELETE /identity/v3/users/8d9d9a125f0d4c4b8e6875104fe228b2 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:35.738077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f8c4c-d2b0-47b2-a5db-71a5df6ec759 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:35.738077 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a90f8c4c-d2b0-47b2-a5db-71a5df6ec759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:35.754259 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f8c4c-d2b0-47b2-a5db-71a5df6ec759 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:35.796106 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:35.815938 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e0e0b-fa0d-440e-991d-3cdc5bc41745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:35.816859 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e0e0b-fa0d-440e-991d-3cdc5bc41745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:35.817137 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e0e0b-fa0d-440e-991d-3cdc5bc41745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:35.817410 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e0e0b-fa0d-440e-991d-3cdc5bc41745 admin admin] PATH_INFO: `/v3/projects/c632a533a9004e48b9dd051d86d4e763` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:35.818612 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e0e0b-fa0d-440e-991d-3cdc5bc41745 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c632a533a9004e48b9dd051d86d4e763)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:35.821721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e0e0b-fa0d-440e-991d-3cdc5bc41745 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:35.960327 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e76e0e0b-fa0d-440e-991d-3cdc5bc41745 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c632a533a9004e48b9dd051d86d4e763', 'request_id': 'req-e76e0e0b-fa0d-440e-991d-3cdc5bc41745'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:35.967656 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e76e0e0b-fa0d-440e-991d-3cdc5bc41745 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c632a533a9004e48b9dd051d86d4e763', 'request_id': 'req-e76e0e0b-fa0d-440e-991d-3cdc5bc41745'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:35.973960 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e76e0e0b-fa0d-440e-991d-3cdc5bc41745 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c632a533a9004e48b9dd051d86d4e763', 'request_id': 'req-e76e0e0b-fa0d-440e-991d-3cdc5bc41745'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:36.093711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e76e0e0b-fa0d-440e-991d-3cdc5bc41745 admin admin] The token cache is being invalidate because project c632a533a9004e48b9dd051d86d4e763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:53:36.094003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e76e0e0b-fa0d-440e-991d-3cdc5bc41745 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:36.094682 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e76e0e0b-fa0d-440e-991d-3cdc5bc41745 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:36.096321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1295/2574] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:53:35 2022] DELETE /identity/v3/projects/c632a533a9004e48b9dd051d86d4e763 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:36.101261 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ea663-27a6-483a-80a2-30b26e0b5062 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:36.101903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ea663-27a6-483a-80a2-30b26e0b5062 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:36.101903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ea663-27a6-483a-80a2-30b26e0b5062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:36.121098 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-3a8ea663-27a6-483a-80a2-30b26e0b5062 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:36.122414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3a8ea663-27a6-483a-80a2-30b26e0b5062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:36.203723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1280/2575] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:36.212547 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd7d0a-b8ec-4570-aafd-6727f2f92111 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:36.215420 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd7d0a-b8ec-4570-aafd-6727f2f92111 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:36.215420 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd7d0a-b8ec-4570-aafd-6727f2f92111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:36.237025 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-49bd7d0a-b8ec-4570-aafd-6727f2f92111 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:36.238393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-49bd7d0a-b8ec-4570-aafd-6727f2f92111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:36.254261 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1296/2576] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:36.255632 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ea663-27a6-483a-80a2-30b26e0b5062 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:36.269121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ea663-27a6-483a-80a2-30b26e0b5062 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:36.275462 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe01eb8-ab29-4b4b-bc9b-80a283308b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:36.276397 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe01eb8-ab29-4b4b-bc9b-80a283308b60 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:36.276609 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe01eb8-ab29-4b4b-bc9b-80a283308b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:36.276915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe01eb8-ab29-4b4b-bc9b-80a283308b60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:36.277698 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe01eb8-ab29-4b4b-bc9b-80a283308b60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:53:36.278943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe01eb8-ab29-4b4b-bc9b-80a283308b60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:36.279930 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe01eb8-ab29-4b4b-bc9b-80a283308b60 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:36.303711 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1281/2577] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:53:36.306655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd7d0a-b8ec-4570-aafd-6727f2f92111 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:36.316856 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd7d0a-b8ec-4570-aafd-6727f2f92111 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:36.323056 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876bf48-27b4-426b-8111-15a057d646a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:36.323773 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876bf48-27b4-426b-8111-15a057d646a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:36.324054 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876bf48-27b4-426b-8111-15a057d646a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:36.324328 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876bf48-27b4-426b-8111-15a057d646a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:36.325060 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876bf48-27b4-426b-8111-15a057d646a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:36.327312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876bf48-27b4-426b-8111-15a057d646a2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:36.443576 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1297/2578] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:53:36 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:36.443990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe01eb8-ab29-4b4b-bc9b-80a283308b60 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:36.455436 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe01eb8-ab29-4b4b-bc9b-80a283308b60 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:36.523414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e343a9-042e-4ff4-968f-a572c44476dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:36.524169 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e343a9-042e-4ff4-968f-a572c44476dd admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:36.524451 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e343a9-042e-4ff4-968f-a572c44476dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:36.524732 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e343a9-042e-4ff4-968f-a572c44476dd admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:36.525454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e343a9-042e-4ff4-968f-a572c44476dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:36.527496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e343a9-042e-4ff4-968f-a572c44476dd admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:36.641325 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1282/2579] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:53:36 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:36.646634 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876bf48-27b4-426b-8111-15a057d646a2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:36.660490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876bf48-27b4-426b-8111-15a057d646a2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:36.665984 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded8fdec-602e-4dcb-9027-6978d9c53d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:36.666659 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8fdec-602e-4dcb-9027-6978d9c53d20 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:36.666946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8fdec-602e-4dcb-9027-6978d9c53d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:36.667225 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8fdec-602e-4dcb-9027-6978d9c53d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:36.668003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded8fdec-602e-4dcb-9027-6978d9c53d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:36.669642 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded8fdec-602e-4dcb-9027-6978d9c53d20 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:36.682120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1298/2580] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:36.686232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e343a9-042e-4ff4-968f-a572c44476dd admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:36.696495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e343a9-042e-4ff4-968f-a572c44476dd admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:36.701529 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe34a5a-45e5-41dd-b1cd-f6fac2346b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:36.702224 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe34a5a-45e5-41dd-b1cd-f6fac2346b78 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:36.702504 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe34a5a-45e5-41dd-b1cd-f6fac2346b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:36.702777 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe34a5a-45e5-41dd-b1cd-f6fac2346b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:36.703503 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe34a5a-45e5-41dd-b1cd-f6fac2346b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:36.705126 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe34a5a-45e5-41dd-b1cd-f6fac2346b78 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:36.719659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1283/2581] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:36.722618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded8fdec-602e-4dcb-9027-6978d9c53d20 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:36.733509 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded8fdec-602e-4dcb-9027-6978d9c53d20 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:36.738588 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de6e90-52f1-4050-833e-93e615787bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:36.740302 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de6e90-52f1-4050-833e-93e615787bf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:36.740302 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de6e90-52f1-4050-833e-93e615787bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:36.740302 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de6e90-52f1-4050-833e-93e615787bf3 admin admin] PATH_INFO: `/v3/projects/b8349cb4d95d404eacfda1328d165e87/users/0b94440ceae044df818f9d1c7e53aea9/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:36.760258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de6e90-52f1-4050-833e-93e615787bf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8349cb4d95d404eacfda1328d165e87, user_id=0b94440ceae044df818f9d1c7e53aea9, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:36.762886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de6e90-52f1-4050-833e-93e615787bf3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:36.860675 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1299/2582] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:53:36 2022] PUT /identity/v3/projects/b8349cb4d95d404eacfda1328d165e87/users/0b94440ceae044df818f9d1c7e53aea9/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:36.872603 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f6908-fb49-4f07-8a87-b16e0bb3d5f0 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:36.872942 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f6908-fb49-4f07-8a87-b16e0bb3d5f0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:36.873182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f6908-fb49-4f07-8a87-b16e0bb3d5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:36.904389 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6a8f6908-fb49-4f07-8a87-b16e0bb3d5f0 None None] MFA Rules not processed for user `0b94440ceae044df818f9d1c7e53aea9`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:36.905601 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6a8f6908-fb49-4f07-8a87-b16e0bb3d5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:36.942449 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:36.981366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1284/2583] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:36.993436 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de6e90-52f1-4050-833e-93e615787bf3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:36.995176 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-24de6e90-52f1-4050-833e-93e615787bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:37.062389 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de6e90-52f1-4050-833e-93e615787bf3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:37.067011 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c682fac-a306-4625-bcdf-4a553b643416 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:37.067716 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c682fac-a306-4625-bcdf-4a553b643416 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:37.068010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c682fac-a306-4625-bcdf-4a553b643416 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:37.068303 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c682fac-a306-4625-bcdf-4a553b643416 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:37.077943 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c682fac-a306-4625-bcdf-4a553b643416 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:37.079467 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c682fac-a306-4625-bcdf-4a553b643416 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:37.088654 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1300/2584] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:53:47.084439 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:47.157847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=12 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:51.174451 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f6908-fb49-4f07-8a87-b16e0bb3d5f0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:51.176243 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6a8f6908-fb49-4f07-8a87-b16e0bb3d5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:51.248687 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f6908-fb49-4f07-8a87-b16e0bb3d5f0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:51.253634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f83db-8480-4791-9323-d81f17bd5b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:51.254558 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f83db-8480-4791-9323-d81f17bd5b87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:51.254839 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f83db-8480-4791-9323-d81f17bd5b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:51.255117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f83db-8480-4791-9323-d81f17bd5b87 admin admin] PATH_INFO: `/v3/users/4e4e1ed088d54ba6a917d02d90283d69` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:51.256398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f83db-8480-4791-9323-d81f17bd5b87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e4e1ed088d54ba6a917d02d90283d69)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:51.258387 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f83db-8480-4791-9323-d81f17bd5b87 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:51.517011 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-860f83db-8480-4791-9323-d81f17bd5b87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e4e1ed088d54ba6a917d02d90283d69'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:51.518139 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-860f83db-8480-4791-9323-d81f17bd5b87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e4e1ed088d54ba6a917d02d90283d69'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:51.518765 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-860f83db-8480-4791-9323-d81f17bd5b87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e4e1ed088d54ba6a917d02d90283d69'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:51.536638 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-860f83db-8480-4791-9323-d81f17bd5b87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e4e1ed088d54ba6a917d02d90283d69'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:51.545195 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-860f83db-8480-4791-9323-d81f17bd5b87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e4e1ed088d54ba6a917d02d90283d69'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:51.620081 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1285/2585] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:53:51 2022] DELETE /identity/v3/users/4e4e1ed088d54ba6a917d02d90283d69 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:51.624807 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca26077-c9af-42e1-bc6e-5222d703abd1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:51.625299 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca26077-c9af-42e1-bc6e-5222d703abd1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:51.625408 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca26077-c9af-42e1-bc6e-5222d703abd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:51.645301 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7ca26077-c9af-42e1-bc6e-5222d703abd1 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:51.646577 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7ca26077-c9af-42e1-bc6e-5222d703abd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:51.710777 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1301/2586] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:51.721186 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f83db-8480-4791-9323-d81f17bd5b87 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:51.722659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-860f83db-8480-4791-9323-d81f17bd5b87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:51.785527 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f83db-8480-4791-9323-d81f17bd5b87 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:51.789506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfc15e-b411-4d78-bf5e-a2ea66802ee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:51.790191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfc15e-b411-4d78-bf5e-a2ea66802ee5 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:51.790477 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfc15e-b411-4d78-bf5e-a2ea66802ee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:51.790753 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfc15e-b411-4d78-bf5e-a2ea66802ee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:51.800503 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cfc15e-b411-4d78-bf5e-a2ea66802ee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:51.801958 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cfc15e-b411-4d78-bf5e-a2ea66802ee5 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:51.812117 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1286/2587] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:53:52.367946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca26077-c9af-42e1-bc6e-5222d703abd1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:52.370369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7ca26077-c9af-42e1-bc6e-5222d703abd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:52.383973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca26077-c9af-42e1-bc6e-5222d703abd1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:52.396204 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a6858-772b-4e05-9571-178601c7f7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:52.396204 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a6858-772b-4e05-9571-178601c7f7b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:52.396204 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a6858-772b-4e05-9571-178601c7f7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:52.396204 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a6858-772b-4e05-9571-178601c7f7b2 admin admin] PATH_INFO: `/v3/projects/025abe7868084bf7acf06f510bd021b5` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:52.397201 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a6858-772b-4e05-9571-178601c7f7b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=025abe7868084bf7acf06f510bd021b5)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:52.399595 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a6858-772b-4e05-9571-178601c7f7b2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:52.557269 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-db1a6858-772b-4e05-9571-178601c7f7b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '025abe7868084bf7acf06f510bd021b5', 'request_id': 'req-db1a6858-772b-4e05-9571-178601c7f7b2'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:52.571065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-db1a6858-772b-4e05-9571-178601c7f7b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '025abe7868084bf7acf06f510bd021b5', 'request_id': 'req-db1a6858-772b-4e05-9571-178601c7f7b2'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:52.581901 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-db1a6858-772b-4e05-9571-178601c7f7b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '025abe7868084bf7acf06f510bd021b5', 'request_id': 'req-db1a6858-772b-4e05-9571-178601c7f7b2'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:52.662260 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-db1a6858-772b-4e05-9571-178601c7f7b2 admin admin] The token cache is being invalidate because project 025abe7868084bf7acf06f510bd021b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:53:52.662842 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-db1a6858-772b-4e05-9571-178601c7f7b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:52.663392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-db1a6858-772b-4e05-9571-178601c7f7b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:52.665354 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1302/2588] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:53:52 2022] DELETE /identity/v3/projects/025abe7868084bf7acf06f510bd021b5 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:52.670085 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfc15e-b411-4d78-bf5e-a2ea66802ee5 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:52.680013 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfc15e-b411-4d78-bf5e-a2ea66802ee5 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:52.700085 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:52.744510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e349632-2101-4552-a043-b30f96537e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:52.745521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e349632-2101-4552-a043-b30f96537e10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:52.745810 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e349632-2101-4552-a043-b30f96537e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:52.746085 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e349632-2101-4552-a043-b30f96537e10 admin admin] PATH_INFO: `/v3/users/c17c83414cfd4eb599f8fff5afe73a3a` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:52.747718 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e349632-2101-4552-a043-b30f96537e10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c17c83414cfd4eb599f8fff5afe73a3a)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:52.749915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e349632-2101-4552-a043-b30f96537e10 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:52.888393 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6e349632-2101-4552-a043-b30f96537e10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c17c83414cfd4eb599f8fff5afe73a3a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:52.889133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6e349632-2101-4552-a043-b30f96537e10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c17c83414cfd4eb599f8fff5afe73a3a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:52.889769 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6e349632-2101-4552-a043-b30f96537e10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c17c83414cfd4eb599f8fff5afe73a3a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:52.902543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6e349632-2101-4552-a043-b30f96537e10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c17c83414cfd4eb599f8fff5afe73a3a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:52.909892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6e349632-2101-4552-a043-b30f96537e10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c17c83414cfd4eb599f8fff5afe73a3a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:53.006469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1287/2589] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:53:52 2022] DELETE /identity/v3/users/c17c83414cfd4eb599f8fff5afe73a3a => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:53:53.562409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a6858-772b-4e05-9571-178601c7f7b2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:53.562409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-db1a6858-772b-4e05-9571-178601c7f7b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:53.583214 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a6858-772b-4e05-9571-178601c7f7b2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:53.646544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e0ee0-4110-4015-b9af-18d6c6721f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:53.648172 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e0ee0-4110-4015-b9af-18d6c6721f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:53.648172 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e0ee0-4110-4015-b9af-18d6c6721f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:53.648172 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e0ee0-4110-4015-b9af-18d6c6721f38 admin admin] PATH_INFO: `/v3/projects/e97e66f048d2408cb7a5d08bab724c3c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:53.649391 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e0ee0-4110-4015-b9af-18d6c6721f38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e97e66f048d2408cb7a5d08bab724c3c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:53.651332 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e0ee0-4110-4015-b9af-18d6c6721f38 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:53.750676 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:53.768069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cc5e0ee0-4110-4015-b9af-18d6c6721f38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e97e66f048d2408cb7a5d08bab724c3c', 'request_id': 'req-cc5e0ee0-4110-4015-b9af-18d6c6721f38'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:53.774133 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cc5e0ee0-4110-4015-b9af-18d6c6721f38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e97e66f048d2408cb7a5d08bab724c3c', 'request_id': 'req-cc5e0ee0-4110-4015-b9af-18d6c6721f38'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:53.781580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cc5e0ee0-4110-4015-b9af-18d6c6721f38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e97e66f048d2408cb7a5d08bab724c3c', 'request_id': 'req-cc5e0ee0-4110-4015-b9af-18d6c6721f38'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:53.861474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cc5e0ee0-4110-4015-b9af-18d6c6721f38 admin admin] The token cache is being invalidate because project e97e66f048d2408cb7a5d08bab724c3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:53:53.861756 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cc5e0ee0-4110-4015-b9af-18d6c6721f38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:53.862571 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cc5e0ee0-4110-4015-b9af-18d6c6721f38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:53.864772 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1303/2590] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:53:53 2022] DELETE /identity/v3/projects/e97e66f048d2408cb7a5d08bab724c3c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:53.872849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86504aad-30c1-4f9d-8234-87e8d942a326 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:53.873698 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86504aad-30c1-4f9d-8234-87e8d942a326 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:53.873979 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86504aad-30c1-4f9d-8234-87e8d942a326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:53.895297 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-86504aad-30c1-4f9d-8234-87e8d942a326 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:53.896637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-86504aad-30c1-4f9d-8234-87e8d942a326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:53.961662 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1288/2591] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:53.970572 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c3b66-561c-49a8-9af8-bb6822965be1 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:53.970872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c3b66-561c-49a8-9af8-bb6822965be1 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:53.971154 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c3b66-561c-49a8-9af8-bb6822965be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:53.990711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d95c3b66-561c-49a8-9af8-bb6822965be1 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:53.991982 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d95c3b66-561c-49a8-9af8-bb6822965be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:54.000198 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1304/2592] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:54.004812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86504aad-30c1-4f9d-8234-87e8d942a326 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:54.016588 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86504aad-30c1-4f9d-8234-87e8d942a326 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:54.021750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74292f-33bd-4db0-8a2e-d07b9b795eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:54.028360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74292f-33bd-4db0-8a2e-d07b9b795eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:54.028360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74292f-33bd-4db0-8a2e-d07b9b795eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:54.028360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74292f-33bd-4db0-8a2e-d07b9b795eb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:54.028360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d74292f-33bd-4db0-8a2e-d07b9b795eb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:53:54.028360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d74292f-33bd-4db0-8a2e-d07b9b795eb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:54.028360 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d74292f-33bd-4db0-8a2e-d07b9b795eb6 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:54.043424 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1289/2593] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:53:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:53:54.044945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c3b66-561c-49a8-9af8-bb6822965be1 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:54.054240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c3b66-561c-49a8-9af8-bb6822965be1 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:54.059027 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58e158-e774-4a12-a869-1feca4163e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:54.059924 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58e158-e774-4a12-a869-1feca4163e4f admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:54.060212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58e158-e774-4a12-a869-1feca4163e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:54.060491 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58e158-e774-4a12-a869-1feca4163e4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:54.061215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58e158-e774-4a12-a869-1feca4163e4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:54.063146 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58e158-e774-4a12-a869-1feca4163e4f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:54.147941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1305/2594] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:53:54 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:54.153637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74292f-33bd-4db0-8a2e-d07b9b795eb6 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:54.164414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74292f-33bd-4db0-8a2e-d07b9b795eb6 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:54.193136 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:54.229403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de7bea-0fa7-47b9-bafe-8676cc019bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:54.231392 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de7bea-0fa7-47b9-bafe-8676cc019bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:54.231392 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de7bea-0fa7-47b9-bafe-8676cc019bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:54.231392 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de7bea-0fa7-47b9-bafe-8676cc019bd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:54.231739 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de7bea-0fa7-47b9-bafe-8676cc019bd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:54.233680 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de7bea-0fa7-47b9-bafe-8676cc019bd6 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:54.364540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1290/2595] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:53:54 2022] POST /identity/v3/users => generated 294 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:54.368214 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58e158-e774-4a12-a869-1feca4163e4f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:54.380105 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58e158-e774-4a12-a869-1feca4163e4f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:54.385051 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ade133-2dd5-4980-933e-d8975ac3bed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:54.385717 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ade133-2dd5-4980-933e-d8975ac3bed0 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:54.386002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ade133-2dd5-4980-933e-d8975ac3bed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:54.386278 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ade133-2dd5-4980-933e-d8975ac3bed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:54.387802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ade133-2dd5-4980-933e-d8975ac3bed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:54.390409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ade133-2dd5-4980-933e-d8975ac3bed0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:54.402366 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1306/2596] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:54.408365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de7bea-0fa7-47b9-bafe-8676cc019bd6 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:54.419031 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de7bea-0fa7-47b9-bafe-8676cc019bd6 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:54.423939 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b66f12-41a8-4fd1-9eda-583f7adabe90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:54.426292 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b66f12-41a8-4fd1-9eda-583f7adabe90 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:54.426292 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b66f12-41a8-4fd1-9eda-583f7adabe90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:54.426292 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b66f12-41a8-4fd1-9eda-583f7adabe90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:54.426292 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b66f12-41a8-4fd1-9eda-583f7adabe90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:54.429967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b66f12-41a8-4fd1-9eda-583f7adabe90 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:54.442606 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1291/2597] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:54.446596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ade133-2dd5-4980-933e-d8975ac3bed0 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:54.467151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ade133-2dd5-4980-933e-d8975ac3bed0 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:54.472316 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3126fcd0-4c5c-492c-87e7-41ade421ff01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:54.473420 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126fcd0-4c5c-492c-87e7-41ade421ff01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:54.473707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126fcd0-4c5c-492c-87e7-41ade421ff01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:54.474093 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126fcd0-4c5c-492c-87e7-41ade421ff01 admin admin] PATH_INFO: `/v3/projects/e01788d015a9482e938cc09c4a9237dc/users/43eca0d51ebd4b8f889322fac7a17395/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:54.490998 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3126fcd0-4c5c-492c-87e7-41ade421ff01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e01788d015a9482e938cc09c4a9237dc, user_id=43eca0d51ebd4b8f889322fac7a17395, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:54.493777 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3126fcd0-4c5c-492c-87e7-41ade421ff01 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:54.583706 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1307/2598] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:53:54 2022] PUT /identity/v3/projects/e01788d015a9482e938cc09c4a9237dc/users/43eca0d51ebd4b8f889322fac7a17395/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:54.594347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1c6e6-3426-4ac0-9cb1-0bdb1b4f4a5b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:54.594648 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1c6e6-3426-4ac0-9cb1-0bdb1b4f4a5b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:54.594929 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1c6e6-3426-4ac0-9cb1-0bdb1b4f4a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:54.621032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a8d1c6e6-3426-4ac0-9cb1-0bdb1b4f4a5b None None] MFA Rules not processed for user `43eca0d51ebd4b8f889322fac7a17395`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:54.622709 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a8d1c6e6-3426-4ac0-9cb1-0bdb1b4f4a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:54.686992 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1292/2599] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:54.691464 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3126fcd0-4c5c-492c-87e7-41ade421ff01 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:54.765458 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3126fcd0-4c5c-492c-87e7-41ade421ff01 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:54.770122 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20669f-3616-4e7f-b736-d1417797be56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:54.770775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20669f-3616-4e7f-b736-d1417797be56 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:54.771060 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20669f-3616-4e7f-b736-d1417797be56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:54.771336 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20669f-3616-4e7f-b736-d1417797be56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:54.772072 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d20669f-3616-4e7f-b736-d1417797be56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:54.774993 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d20669f-3616-4e7f-b736-d1417797be56 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:54.868403 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1308/2600] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:53:54 2022] POST /identity/v3/projects => generated 387 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:54.873438 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1c6e6-3426-4ac0-9cb1-0bdb1b4f4a5b None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:54.884337 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1c6e6-3426-4ac0-9cb1-0bdb1b4f4a5b None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:54.942964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad81c6-2e2b-4f38-8366-c79212577f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:54.943720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad81c6-2e2b-4f38-8366-c79212577f98 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:54.944032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad81c6-2e2b-4f38-8366-c79212577f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:54.944314 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad81c6-2e2b-4f38-8366-c79212577f98 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:54.945015 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad81c6-2e2b-4f38-8366-c79212577f98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:54.947006 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad81c6-2e2b-4f38-8366-c79212577f98 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:55.032368 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1293/2601] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:53:54 2022] POST /identity/v3/users => generated 300 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:55.036253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20669f-3616-4e7f-b736-d1417797be56 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:55.048018 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20669f-3616-4e7f-b736-d1417797be56 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:55.052853 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd778666-3632-4053-ade5-6613d3624e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:55.053514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd778666-3632-4053-ade5-6613d3624e84 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:55.053801 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd778666-3632-4053-ade5-6613d3624e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:55.054086 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd778666-3632-4053-ade5-6613d3624e84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:55.054814 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd778666-3632-4053-ade5-6613d3624e84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:55.056503 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd778666-3632-4053-ade5-6613d3624e84 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:55.068300 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1309/2602] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:55.072586 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad81c6-2e2b-4f38-8366-c79212577f98 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:55.083819 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad81c6-2e2b-4f38-8366-c79212577f98 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:55.088403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbc7b0-1052-4525-8c19-4d2e788efb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:55.089634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbc7b0-1052-4525-8c19-4d2e788efb44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:55.089634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbc7b0-1052-4525-8c19-4d2e788efb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:55.089894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbc7b0-1052-4525-8c19-4d2e788efb44 admin admin] PATH_INFO: `/v3/projects/1f7063fdc72240daad11c39d928157e8/users/cb947f8b466b4201be37228bc26c0c5a/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:55.105990 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adbc7b0-1052-4525-8c19-4d2e788efb44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f7063fdc72240daad11c39d928157e8, user_id=cb947f8b466b4201be37228bc26c0c5a, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:55.108683 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adbc7b0-1052-4525-8c19-4d2e788efb44 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:55.175190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1294/2603] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:53:55 2022] PUT /identity/v3/projects/1f7063fdc72240daad11c39d928157e8/users/cb947f8b466b4201be37228bc26c0c5a/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:55.179213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd778666-3632-4053-ade5-6613d3624e84 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:55.189732 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd778666-3632-4053-ade5-6613d3624e84 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:55.212138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:55.250405 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a1511-427b-4521-9234-a31a50114cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:55.251082 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a1511-427b-4521-9234-a31a50114cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:55.251370 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a1511-427b-4521-9234-a31a50114cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:55.251649 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a1511-427b-4521-9234-a31a50114cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:55.252411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a1511-427b-4521-9234-a31a50114cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:55.254151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a1511-427b-4521-9234-a31a50114cf8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:55.265454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1310/2604] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:55.269791 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbc7b0-1052-4525-8c19-4d2e788efb44 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:55.275243 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:55.281081 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbc7b0-1052-4525-8c19-4d2e788efb44 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:55.285830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3a97e-dda7-4f45-a074-5751625dc4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:55.286712 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3a97e-dda7-4f45-a074-5751625dc4ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:55.287075 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3a97e-dda7-4f45-a074-5751625dc4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:55.287354 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3a97e-dda7-4f45-a074-5751625dc4ee admin admin] PATH_INFO: `/v3/projects/1f7063fdc72240daad11c39d928157e8/users/cb947f8b466b4201be37228bc26c0c5a/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:55.289387 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3a97e-dda7-4f45-a074-5751625dc4ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f7063fdc72240daad11c39d928157e8, user_id=cb947f8b466b4201be37228bc26c0c5a, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:55.292053 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3a97e-dda7-4f45-a074-5751625dc4ee admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:55.302540 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1295/2605] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:53:55 2022] PUT /identity/v3/projects/1f7063fdc72240daad11c39d928157e8/users/cb947f8b466b4201be37228bc26c0c5a/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:55.313825 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242cf91-d8e6-4db6-a7b9-aceafd8debe2 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:55.314129 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242cf91-d8e6-4db6-a7b9-aceafd8debe2 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:55.314417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242cf91-d8e6-4db6-a7b9-aceafd8debe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:55.342020 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-9242cf91-d8e6-4db6-a7b9-aceafd8debe2 None None] MFA Rules not processed for user `cb947f8b466b4201be37228bc26c0c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:55.343220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9242cf91-d8e6-4db6-a7b9-aceafd8debe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:55.410709 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1311/2606] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:55.422410 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3a97e-dda7-4f45-a074-5751625dc4ee admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:55.424132 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-50e3a97e-dda7-4f45-a074-5751625dc4ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:55.482901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3a97e-dda7-4f45-a074-5751625dc4ee admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:55.486578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34775bfd-de5d-4b1d-a924-599972284ffb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:55.487217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34775bfd-de5d-4b1d-a924-599972284ffb service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:55.487501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34775bfd-de5d-4b1d-a924-599972284ffb service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:55.487806 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34775bfd-de5d-4b1d-a924-599972284ffb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:55.497292 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34775bfd-de5d-4b1d-a924-599972284ffb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:55.498720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34775bfd-de5d-4b1d-a924-599972284ffb service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:55.508182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1296/2607] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:53:55.536998 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242cf91-d8e6-4db6-a7b9-aceafd8debe2 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:55.599682 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242cf91-d8e6-4db6-a7b9-aceafd8debe2 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:55.604290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980e6b9-eb49-4e38-bff2-f9fe7c19c044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:55.605108 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980e6b9-eb49-4e38-bff2-f9fe7c19c044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:55.605388 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980e6b9-eb49-4e38-bff2-f9fe7c19c044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:55.605660 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980e6b9-eb49-4e38-bff2-f9fe7c19c044 admin admin] PATH_INFO: `/v3/users/43eca0d51ebd4b8f889322fac7a17395` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:55.607146 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980e6b9-eb49-4e38-bff2-f9fe7c19c044 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43eca0d51ebd4b8f889322fac7a17395)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:55.609079 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980e6b9-eb49-4e38-bff2-f9fe7c19c044 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:55.830925 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b980e6b9-eb49-4e38-bff2-f9fe7c19c044 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43eca0d51ebd4b8f889322fac7a17395'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:55.831622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b980e6b9-eb49-4e38-bff2-f9fe7c19c044 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43eca0d51ebd4b8f889322fac7a17395'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:55.832189 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b980e6b9-eb49-4e38-bff2-f9fe7c19c044 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43eca0d51ebd4b8f889322fac7a17395'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:55.833366 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:55.845460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b980e6b9-eb49-4e38-bff2-f9fe7c19c044 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43eca0d51ebd4b8f889322fac7a17395'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:55.852428 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b980e6b9-eb49-4e38-bff2-f9fe7c19c044 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43eca0d51ebd4b8f889322fac7a17395'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:55.938229 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1312/2608] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:53:55 2022] DELETE /identity/v3/users/43eca0d51ebd4b8f889322fac7a17395 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:55.942777 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0b0fc-8da2-483b-9f92-fb5f627490ef None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:55.943266 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0b0fc-8da2-483b-9f92-fb5f627490ef None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:55.943361 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0b0fc-8da2-483b-9f92-fb5f627490ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:55.960434 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d7e0b0fc-8da2-483b-9f92-fb5f627490ef None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:55.961629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d7e0b0fc-8da2-483b-9f92-fb5f627490ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:56.148455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1297/2609] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:56.158249 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980e6b9-eb49-4e38-bff2-f9fe7c19c044 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:56.159798 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b980e6b9-eb49-4e38-bff2-f9fe7c19c044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:56.219558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980e6b9-eb49-4e38-bff2-f9fe7c19c044 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:56.223338 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212b6bc-a590-4461-b301-d341ac86f097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:56.224025 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212b6bc-a590-4461-b301-d341ac86f097 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:56.224310 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212b6bc-a590-4461-b301-d341ac86f097 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:56.224586 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212b6bc-a590-4461-b301-d341ac86f097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:56.235047 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212b6bc-a590-4461-b301-d341ac86f097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:56.236522 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212b6bc-a590-4461-b301-d341ac86f097 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:56.246844 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1313/2610] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:53:56.722639 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0b0fc-8da2-483b-9f92-fb5f627490ef None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:56.724284 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d7e0b0fc-8da2-483b-9f92-fb5f627490ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:56.736607 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0b0fc-8da2-483b-9f92-fb5f627490ef None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:56.741871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4167b54-8a9a-404b-8f1a-1b2be9708c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:56.742750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4167b54-8a9a-404b-8f1a-1b2be9708c04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:56.743060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4167b54-8a9a-404b-8f1a-1b2be9708c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:56.743302 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4167b54-8a9a-404b-8f1a-1b2be9708c04 admin admin] PATH_INFO: `/v3/projects/e01788d015a9482e938cc09c4a9237dc` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:56.744644 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4167b54-8a9a-404b-8f1a-1b2be9708c04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e01788d015a9482e938cc09c4a9237dc)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:56.746577 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4167b54-8a9a-404b-8f1a-1b2be9708c04 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:56.850154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c4167b54-8a9a-404b-8f1a-1b2be9708c04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e01788d015a9482e938cc09c4a9237dc', 'request_id': 'req-c4167b54-8a9a-404b-8f1a-1b2be9708c04'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:56.855755 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c4167b54-8a9a-404b-8f1a-1b2be9708c04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e01788d015a9482e938cc09c4a9237dc', 'request_id': 'req-c4167b54-8a9a-404b-8f1a-1b2be9708c04'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:56.862219 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c4167b54-8a9a-404b-8f1a-1b2be9708c04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e01788d015a9482e938cc09c4a9237dc', 'request_id': 'req-c4167b54-8a9a-404b-8f1a-1b2be9708c04'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:56.934528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c4167b54-8a9a-404b-8f1a-1b2be9708c04 admin admin] The token cache is being invalidate because project e01788d015a9482e938cc09c4a9237dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:53:56.934841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c4167b54-8a9a-404b-8f1a-1b2be9708c04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:56.935548 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-c4167b54-8a9a-404b-8f1a-1b2be9708c04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:56.937191 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1298/2611] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:53:56 2022] DELETE /identity/v3/projects/e01788d015a9482e938cc09c4a9237dc => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:56.941275 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212b6bc-a590-4461-b301-d341ac86f097 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:56.952445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212b6bc-a590-4461-b301-d341ac86f097 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:57.010856 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12c837-542d-49a5-9259-3e9295aec8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:57.011656 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12c837-542d-49a5-9259-3e9295aec8ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:57.011961 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12c837-542d-49a5-9259-3e9295aec8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:57.012311 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12c837-542d-49a5-9259-3e9295aec8ef admin admin] PATH_INFO: `/v3/users/cb947f8b466b4201be37228bc26c0c5a` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:57.013883 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12c837-542d-49a5-9259-3e9295aec8ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb947f8b466b4201be37228bc26c0c5a)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:57.015836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12c837-542d-49a5-9259-3e9295aec8ef admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:57.234561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cf12c837-542d-49a5-9259-3e9295aec8ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb947f8b466b4201be37228bc26c0c5a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:57.235210 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cf12c837-542d-49a5-9259-3e9295aec8ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb947f8b466b4201be37228bc26c0c5a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:57.235841 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cf12c837-542d-49a5-9259-3e9295aec8ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb947f8b466b4201be37228bc26c0c5a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:57.248385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cf12c837-542d-49a5-9259-3e9295aec8ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb947f8b466b4201be37228bc26c0c5a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:57.252892 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:57.255932 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cf12c837-542d-49a5-9259-3e9295aec8ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb947f8b466b4201be37228bc26c0c5a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:57.351774 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1314/2612] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:53:56 2022] DELETE /identity/v3/users/cb947f8b466b4201be37228bc26c0c5a => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:57.819659 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4167b54-8a9a-404b-8f1a-1b2be9708c04 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:57.821309 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c4167b54-8a9a-404b-8f1a-1b2be9708c04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:57.845823 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4167b54-8a9a-404b-8f1a-1b2be9708c04 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:57.860534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:57.913024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df780d8-16d6-4f59-a7fe-a0ae9de83284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:57.913816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df780d8-16d6-4f59-a7fe-a0ae9de83284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:57.914102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df780d8-16d6-4f59-a7fe-a0ae9de83284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:57.914377 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df780d8-16d6-4f59-a7fe-a0ae9de83284 admin admin] PATH_INFO: `/v3/projects/1f7063fdc72240daad11c39d928157e8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:57.915686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df780d8-16d6-4f59-a7fe-a0ae9de83284 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f7063fdc72240daad11c39d928157e8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:57.917660 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df780d8-16d6-4f59-a7fe-a0ae9de83284 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:58.046682 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4df780d8-16d6-4f59-a7fe-a0ae9de83284 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f7063fdc72240daad11c39d928157e8', 'request_id': 'req-4df780d8-16d6-4f59-a7fe-a0ae9de83284'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:58.052052 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4df780d8-16d6-4f59-a7fe-a0ae9de83284 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f7063fdc72240daad11c39d928157e8', 'request_id': 'req-4df780d8-16d6-4f59-a7fe-a0ae9de83284'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:58.058081 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4df780d8-16d6-4f59-a7fe-a0ae9de83284 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f7063fdc72240daad11c39d928157e8', 'request_id': 'req-4df780d8-16d6-4f59-a7fe-a0ae9de83284'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:58.147878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4df780d8-16d6-4f59-a7fe-a0ae9de83284 admin admin] The token cache is being invalidate because project 1f7063fdc72240daad11c39d928157e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:53:58.148164 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4df780d8-16d6-4f59-a7fe-a0ae9de83284 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:58.148786 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4df780d8-16d6-4f59-a7fe-a0ae9de83284 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:53:58.150380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1299/2613] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:53:57 2022] DELETE /identity/v3/projects/1f7063fdc72240daad11c39d928157e8 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:53:58.155643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f4bfc-1b7c-4dcd-ac57-672dc85d3d4c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:58.156276 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f4bfc-1b7c-4dcd-ac57-672dc85d3d4c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:58.156276 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f4bfc-1b7c-4dcd-ac57-672dc85d3d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:58.176583 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-4a6f4bfc-1b7c-4dcd-ac57-672dc85d3d4c None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:58.177718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-4a6f4bfc-1b7c-4dcd-ac57-672dc85d3d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:58.250827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1315/2614] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:58.261975 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1c89c-c52a-4b8b-ace5-0d81faf9112b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:58.262267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1c89c-c52a-4b8b-ace5-0d81faf9112b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:58.263174 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1c89c-c52a-4b8b-ace5-0d81faf9112b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:58.282639 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-12b1c89c-c52a-4b8b-ace5-0d81faf9112b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:58.283912 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-12b1c89c-c52a-4b8b-ace5-0d81faf9112b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:58.292446 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1300/2615] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:58.299242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f4bfc-1b7c-4dcd-ac57-672dc85d3d4c None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:58.309883 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f4bfc-1b7c-4dcd-ac57-672dc85d3d4c None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:58.314528 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96284963-e5cb-4f73-8302-ede0339f81c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:58.315183 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96284963-e5cb-4f73-8302-ede0339f81c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:58.315470 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96284963-e5cb-4f73-8302-ede0339f81c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:58.315785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96284963-e5cb-4f73-8302-ede0339f81c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:58.316522 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96284963-e5cb-4f73-8302-ede0339f81c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:53:58.316827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96284963-e5cb-4f73-8302-ede0339f81c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:58.318473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96284963-e5cb-4f73-8302-ede0339f81c3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:58.330960 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1316/2616] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:53:58.335586 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1c89c-c52a-4b8b-ace5-0d81faf9112b None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:58.345318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1c89c-c52a-4b8b-ace5-0d81faf9112b None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:58.350142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ceb2b4-24e3-4a42-a453-c90cc38a836a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:58.350787 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ceb2b4-24e3-4a42-a453-c90cc38a836a admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:58.351065 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ceb2b4-24e3-4a42-a453-c90cc38a836a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:58.351337 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ceb2b4-24e3-4a42-a453-c90cc38a836a admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:58.353473 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ceb2b4-24e3-4a42-a453-c90cc38a836a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:58.353931 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ceb2b4-24e3-4a42-a453-c90cc38a836a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:58.430093 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1301/2617] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:53:58 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:58.435831 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96284963-e5cb-4f73-8302-ede0339f81c3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:58.445691 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96284963-e5cb-4f73-8302-ede0339f81c3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:58.502478 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494aed5-b9ed-42bb-9948-7c0a65eeba66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:58.503143 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494aed5-b9ed-42bb-9948-7c0a65eeba66 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:58.503425 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494aed5-b9ed-42bb-9948-7c0a65eeba66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:58.503727 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494aed5-b9ed-42bb-9948-7c0a65eeba66 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:58.504411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494aed5-b9ed-42bb-9948-7c0a65eeba66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:58.506330 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494aed5-b9ed-42bb-9948-7c0a65eeba66 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:58.577747 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1317/2618] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:53:58 2022] POST /identity/v3/users => generated 287 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:53:58.581968 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ceb2b4-24e3-4a42-a453-c90cc38a836a admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:58.593050 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ceb2b4-24e3-4a42-a453-c90cc38a836a admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:58.597606 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2a7c1-5cf9-4a52-89ed-d2ee1a288eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:58.598250 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2a7c1-5cf9-4a52-89ed-d2ee1a288eab admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:58.598530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2a7c1-5cf9-4a52-89ed-d2ee1a288eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:58.598804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2a7c1-5cf9-4a52-89ed-d2ee1a288eab admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:58.599518 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a2a7c1-5cf9-4a52-89ed-d2ee1a288eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:58.601122 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a2a7c1-5cf9-4a52-89ed-d2ee1a288eab admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:58.612555 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1302/2619] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:58.616524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494aed5-b9ed-42bb-9948-7c0a65eeba66 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:58.626835 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494aed5-b9ed-42bb-9948-7c0a65eeba66 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:58.631605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a22774-31e8-455d-be6f-0e03e0705af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:58.632298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a22774-31e8-455d-be6f-0e03e0705af9 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:58.632580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a22774-31e8-455d-be6f-0e03e0705af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:58.632858 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a22774-31e8-455d-be6f-0e03e0705af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:58.633584 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a22774-31e8-455d-be6f-0e03e0705af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:58.635190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a22774-31e8-455d-be6f-0e03e0705af9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:58.647011 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1318/2620] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:53:58.651079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2a7c1-5cf9-4a52-89ed-d2ee1a288eab admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:58.662591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2a7c1-5cf9-4a52-89ed-d2ee1a288eab admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:58.667266 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9496a-bbba-4351-bb74-3214a7659392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:58.668183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9496a-bbba-4351-bb74-3214a7659392 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:58.669099 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9496a-bbba-4351-bb74-3214a7659392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:58.669605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9496a-bbba-4351-bb74-3214a7659392 admin admin] PATH_INFO: `/v3/projects/401f21125b3146fe9deba3df1abf1803/users/4e8cc08181c74bb584fdca44adba662a/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:58.684739 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9496a-bbba-4351-bb74-3214a7659392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=401f21125b3146fe9deba3df1abf1803, user_id=4e8cc08181c74bb584fdca44adba662a, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:58.687496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9496a-bbba-4351-bb74-3214a7659392 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:58.756730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1303/2621] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:53:58 2022] PUT /identity/v3/projects/401f21125b3146fe9deba3df1abf1803/users/4e8cc08181c74bb584fdca44adba662a/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:53:58.761816 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db2114-31d5-4963-874d-16aa66cfe59c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:58.762110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db2114-31d5-4963-874d-16aa66cfe59c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:58.762443 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db2114-31d5-4963-874d-16aa66cfe59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:58.780419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-66db2114-31d5-4963-874d-16aa66cfe59c None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:53:58.781977 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-66db2114-31d5-4963-874d-16aa66cfe59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:58.829951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:58.846473 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1319/2622] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:53:58.853165 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9496a-bbba-4351-bb74-3214a7659392 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:53:58.857728 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-66f9496a-bbba-4351-bb74-3214a7659392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:53:58.906309 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:53:58.918383 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9496a-bbba-4351-bb74-3214a7659392 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:53:58.923187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1e73b-acc7-42b7-abd5-e11a3bbcead1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:53:58.923878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1e73b-acc7-42b7-abd5-e11a3bbcead1 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:53:58.924166 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1e73b-acc7-42b7-abd5-e11a3bbcead1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:53:58.924442 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1e73b-acc7-42b7-abd5-e11a3bbcead1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:53:58.934135 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1e73b-acc7-42b7-abd5-e11a3bbcead1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:53:58.936222 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1e73b-acc7-42b7-abd5-e11a3bbcead1 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:53:58.945622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1304/2623] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:53:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:54:00.244134 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b2fa2-8aa8-4597-9ea0-713aad8fe58e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:00.244617 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b2fa2-8aa8-4597-9ea0-713aad8fe58e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:00.244711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b2fa2-8aa8-4597-9ea0-713aad8fe58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:00.275146 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-644b2fa2-8aa8-4597-9ea0-713aad8fe58e None None] MFA Rules not processed for user `4e8cc08181c74bb584fdca44adba662a`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:00.276404 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-644b2fa2-8aa8-4597-9ea0-713aad8fe58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:00.347405 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1320/2624] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:00.359892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1e73b-acc7-42b7-abd5-e11a3bbcead1 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:00.361546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-86f1e73b-acc7-42b7-abd5-e11a3bbcead1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:00.419712 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1e73b-acc7-42b7-abd5-e11a3bbcead1 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:00.424004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784b8aa-b2b8-4506-964a-1e11a90b6faf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:00.424774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784b8aa-b2b8-4506-964a-1e11a90b6faf service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:00.425058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784b8aa-b2b8-4506-964a-1e11a90b6faf service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:00.425331 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784b8aa-b2b8-4506-964a-1e11a90b6faf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:00.437766 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9784b8aa-b2b8-4506-964a-1e11a90b6faf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:00.439261 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9784b8aa-b2b8-4506-964a-1e11a90b6faf service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:00.448776 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1305/2625] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:54:00.515833 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b2fa2-8aa8-4597-9ea0-713aad8fe58e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:00.527313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b2fa2-8aa8-4597-9ea0-713aad8fe58e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:00.531740 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c8d17-fbca-444b-8888-73d775b6bf4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:00.532396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c8d17-fbca-444b-8888-73d775b6bf4a service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:00.532675 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c8d17-fbca-444b-8888-73d775b6bf4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:00.532950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c8d17-fbca-444b-8888-73d775b6bf4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:00.535021 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a26c8d17-fbca-444b-8888-73d775b6bf4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:00.551372 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c8d17-fbca-444b-8888-73d775b6bf4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:00.552984 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c8d17-fbca-444b-8888-73d775b6bf4a service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:00.565220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1321/2626] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:54:02.288295 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725436b8-140a-4bd4-8dc8-f0ebf2528e92 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:02.289097 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725436b8-140a-4bd4-8dc8-f0ebf2528e92 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:02.289666 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725436b8-140a-4bd4-8dc8-f0ebf2528e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:02.321813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-725436b8-140a-4bd4-8dc8-f0ebf2528e92 None None] MFA Rules not processed for user `4e8cc08181c74bb584fdca44adba662a`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:02.324118 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-725436b8-140a-4bd4-8dc8-f0ebf2528e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:02.334058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1306/2627] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:03.609006 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c8d17-fbca-444b-8888-73d775b6bf4a service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:03.610447 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a26c8d17-fbca-444b-8888-73d775b6bf4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:03.623811 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c8d17-fbca-444b-8888-73d775b6bf4a service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:03.630361 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839fb3c-b4aa-44a7-a960-e3607cc7b82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:03.632907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839fb3c-b4aa-44a7-a960-e3607cc7b82a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:03.633417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839fb3c-b4aa-44a7-a960-e3607cc7b82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:03.644030 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839fb3c-b4aa-44a7-a960-e3607cc7b82a admin admin] PATH_INFO: `/v3/users/0b94440ceae044df818f9d1c7e53aea9` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:03.650722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839fb3c-b4aa-44a7-a960-e3607cc7b82a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b94440ceae044df818f9d1c7e53aea9)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:03.657948 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839fb3c-b4aa-44a7-a960-e3607cc7b82a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:03.856612 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9839fb3c-b4aa-44a7-a960-e3607cc7b82a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b94440ceae044df818f9d1c7e53aea9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:03.858257 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9839fb3c-b4aa-44a7-a960-e3607cc7b82a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b94440ceae044df818f9d1c7e53aea9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:03.859491 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9839fb3c-b4aa-44a7-a960-e3607cc7b82a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b94440ceae044df818f9d1c7e53aea9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:03.874154 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9839fb3c-b4aa-44a7-a960-e3607cc7b82a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b94440ceae044df818f9d1c7e53aea9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:03.881713 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9839fb3c-b4aa-44a7-a960-e3607cc7b82a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b94440ceae044df818f9d1c7e53aea9'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:04.006375 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1322/2628] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:54:03 2022] DELETE /identity/v3/users/0b94440ceae044df818f9d1c7e53aea9 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:04.010954 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b917a-487f-490b-94a2-f792e75b7cec None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:04.011492 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b917a-487f-490b-94a2-f792e75b7cec None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:04.011661 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b917a-487f-490b-94a2-f792e75b7cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:04.036225 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-961b917a-487f-490b-94a2-f792e75b7cec None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:04.037617 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-961b917a-487f-490b-94a2-f792e75b7cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:04.104469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1307/2629] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:54:04.114655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839fb3c-b4aa-44a7-a960-e3607cc7b82a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:04.116299 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9839fb3c-b4aa-44a7-a960-e3607cc7b82a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:04.175337 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:04.193121 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839fb3c-b4aa-44a7-a960-e3607cc7b82a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:04.197386 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea806dc-67d2-4944-84e3-8f2ee8bff5a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:04.198121 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea806dc-67d2-4944-84e3-8f2ee8bff5a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:04.198378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea806dc-67d2-4944-84e3-8f2ee8bff5a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:04.198655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea806dc-67d2-4944-84e3-8f2ee8bff5a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:04.215418 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea806dc-67d2-4944-84e3-8f2ee8bff5a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:04.217648 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea806dc-67d2-4944-84e3-8f2ee8bff5a5 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:04.234769 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1323/2630] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:54:04.760454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b917a-487f-490b-94a2-f792e75b7cec None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:04.762046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-961b917a-487f-490b-94a2-f792e75b7cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:04.776269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b917a-487f-490b-94a2-f792e75b7cec None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:04.782238 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e753d68-eb2d-4d32-bfb1-25b45100edbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:04.783120 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e753d68-eb2d-4d32-bfb1-25b45100edbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:04.783487 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e753d68-eb2d-4d32-bfb1-25b45100edbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:04.783812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e753d68-eb2d-4d32-bfb1-25b45100edbc admin admin] PATH_INFO: `/v3/projects/b8349cb4d95d404eacfda1328d165e87` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:04.786983 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e753d68-eb2d-4d32-bfb1-25b45100edbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8349cb4d95d404eacfda1328d165e87)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:04.789366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e753d68-eb2d-4d32-bfb1-25b45100edbc admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:04.944339 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1e753d68-eb2d-4d32-bfb1-25b45100edbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8349cb4d95d404eacfda1328d165e87', 'request_id': 'req-1e753d68-eb2d-4d32-bfb1-25b45100edbc'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:04.951004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1e753d68-eb2d-4d32-bfb1-25b45100edbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8349cb4d95d404eacfda1328d165e87', 'request_id': 'req-1e753d68-eb2d-4d32-bfb1-25b45100edbc'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:04.957510 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1e753d68-eb2d-4d32-bfb1-25b45100edbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8349cb4d95d404eacfda1328d165e87', 'request_id': 'req-1e753d68-eb2d-4d32-bfb1-25b45100edbc'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:05.080631 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea806dc-67d2-4944-84e3-8f2ee8bff5a5 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:05.082124 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8ea806dc-67d2-4944-84e3-8f2ee8bff5a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:05.096496 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea806dc-67d2-4944-84e3-8f2ee8bff5a5 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:05.106573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1e753d68-eb2d-4d32-bfb1-25b45100edbc admin admin] The token cache is being invalidate because project b8349cb4d95d404eacfda1328d165e87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:54:05.107174 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1e753d68-eb2d-4d32-bfb1-25b45100edbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:05.107511 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1e753d68-eb2d-4d32-bfb1-25b45100edbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:05.109370 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1308/2631] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:54:04 2022] DELETE /identity/v3/projects/b8349cb4d95d404eacfda1328d165e87 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:05.114427 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41729fcd-c04e-4547-867f-8985b460f84d None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:05.114785 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41729fcd-c04e-4547-867f-8985b460f84d None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:05.115129 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41729fcd-c04e-4547-867f-8985b460f84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:05.126567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:05.138271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-41729fcd-c04e-4547-867f-8985b460f84d None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:05.140359 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-41729fcd-c04e-4547-867f-8985b460f84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:05.162114 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29753567-5d03-4631-b3f7-caea54397447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:05.163079 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29753567-5d03-4631-b3f7-caea54397447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:05.163272 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29753567-5d03-4631-b3f7-caea54397447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:05.163547 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29753567-5d03-4631-b3f7-caea54397447 admin admin] PATH_INFO: `/v3/users/88b4bd52ede644f0aac8dff94e09dd7d` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:05.164860 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29753567-5d03-4631-b3f7-caea54397447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88b4bd52ede644f0aac8dff94e09dd7d)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:05.166952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29753567-5d03-4631-b3f7-caea54397447 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:05.206583 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1309/2632] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:54:05.214244 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fea5f-1df5-4aed-92fd-6e8b2c90bcb1 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:05.214530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fea5f-1df5-4aed-92fd-6e8b2c90bcb1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:05.214803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fea5f-1df5-4aed-92fd-6e8b2c90bcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:05.233976 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-326fea5f-1df5-4aed-92fd-6e8b2c90bcb1 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:05.235297 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-326fea5f-1df5-4aed-92fd-6e8b2c90bcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:05.243690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1310/2633] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:54:05.248865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326fea5f-1df5-4aed-92fd-6e8b2c90bcb1 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:05.259446 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326fea5f-1df5-4aed-92fd-6e8b2c90bcb1 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:05.265594 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3b6c2-d062-44ec-8a30-72258cbcebd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:05.266271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3b6c2-d062-44ec-8a30-72258cbcebd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:05.266558 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3b6c2-d062-44ec-8a30-72258cbcebd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:05.266835 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3b6c2-d062-44ec-8a30-72258cbcebd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:05.267644 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3b6c2-d062-44ec-8a30-72258cbcebd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:54:05.267983 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3b6c2-d062-44ec-8a30-72258cbcebd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:05.269582 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3b6c2-d062-44ec-8a30-72258cbcebd3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:05.284411 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1311/2634] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:54:05.288794 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3b6c2-d062-44ec-8a30-72258cbcebd3 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:05.299723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3b6c2-d062-44ec-8a30-72258cbcebd3 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:05.304390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019075e-04e3-4f9d-ad5d-cf1c537278cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:05.305040 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019075e-04e3-4f9d-ad5d-cf1c537278cd admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:05.305326 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019075e-04e3-4f9d-ad5d-cf1c537278cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:05.305601 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019075e-04e3-4f9d-ad5d-cf1c537278cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:05.306295 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019075e-04e3-4f9d-ad5d-cf1c537278cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:05.308309 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019075e-04e3-4f9d-ad5d-cf1c537278cd admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:05.395328 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-29753567-5d03-4631-b3f7-caea54397447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88b4bd52ede644f0aac8dff94e09dd7d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:05.396092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-29753567-5d03-4631-b3f7-caea54397447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88b4bd52ede644f0aac8dff94e09dd7d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:05.396970 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-29753567-5d03-4631-b3f7-caea54397447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88b4bd52ede644f0aac8dff94e09dd7d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:05.410632 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-29753567-5d03-4631-b3f7-caea54397447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88b4bd52ede644f0aac8dff94e09dd7d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:05.415628 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1312/2635] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:54:05 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:54:05.420581 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-29753567-5d03-4631-b3f7-caea54397447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88b4bd52ede644f0aac8dff94e09dd7d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:05.420716 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019075e-04e3-4f9d-ad5d-cf1c537278cd admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:05.421537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4019075e-04e3-4f9d-ad5d-cf1c537278cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:05.484663 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019075e-04e3-4f9d-ad5d-cf1c537278cd admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:05.489532 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7948661-e25a-477e-b63a-0b80fb765de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:05.490206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7948661-e25a-477e-b63a-0b80fb765de0 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:05.490491 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7948661-e25a-477e-b63a-0b80fb765de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:05.490797 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7948661-e25a-477e-b63a-0b80fb765de0 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:05.491488 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7948661-e25a-477e-b63a-0b80fb765de0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:05.493507 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7948661-e25a-477e-b63a-0b80fb765de0 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:05.508799 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1324/2636] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:54:05 2022] DELETE /identity/v3/users/88b4bd52ede644f0aac8dff94e09dd7d => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:54:05.558852 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29753567-5d03-4631-b3f7-caea54397447 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:05.561335 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-29753567-5d03-4631-b3f7-caea54397447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:05.573881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1313/2637] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:54:05 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:54:05.584847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7948661-e25a-477e-b63a-0b80fb765de0 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:05.599322 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7948661-e25a-477e-b63a-0b80fb765de0 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:05.641219 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:05.677968 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1c105-ce05-489a-93ef-8a8d6a0bf5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:05.679327 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1c105-ce05-489a-93ef-8a8d6a0bf5ff admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:05.679858 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1c105-ce05-489a-93ef-8a8d6a0bf5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:05.680321 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1c105-ce05-489a-93ef-8a8d6a0bf5ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:05.681570 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1c105-ce05-489a-93ef-8a8d6a0bf5ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:05.684129 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1c105-ce05-489a-93ef-8a8d6a0bf5ff admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:05.690872 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:05.691566 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29753567-5d03-4631-b3f7-caea54397447 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:05.702148 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a098a0-c5e1-4912-adc2-80ea1f65f5fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd606fd7ef269460a8110223932f02d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:05.703172 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a098a0-c5e1-4912-adc2-80ea1f65f5fa service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:05.703631 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a098a0-c5e1-4912-adc2-80ea1f65f5fa service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:05.704124 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a098a0-c5e1-4912-adc2-80ea1f65f5fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:05.707696 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1314/2638] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:54:05.707973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-68a098a0-c5e1-4912-adc2-80ea1f65f5fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:05.708924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1c105-ce05-489a-93ef-8a8d6a0bf5ff admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:05.718845 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1c105-ce05-489a-93ef-8a8d6a0bf5ff admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:05.724645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a098a0-c5e1-4912-adc2-80ea1f65f5fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:05.724645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a098a0-c5e1-4912-adc2-80ea1f65f5fa service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:05.724867 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97f8a5-f412-4ca3-b567-b5ed9151d823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:05.724867 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97f8a5-f412-4ca3-b567-b5ed9151d823 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:05.724867 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97f8a5-f412-4ca3-b567-b5ed9151d823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:05.724867 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97f8a5-f412-4ca3-b567-b5ed9151d823 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:05.725604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97f8a5-f412-4ca3-b567-b5ed9151d823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:05.727178 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97f8a5-f412-4ca3-b567-b5ed9151d823 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:05.731153 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1325/2639] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:54:05.738863 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1315/2640] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:54:05.743106 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a098a0-c5e1-4912-adc2-80ea1f65f5fa service placement] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:05.756643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a098a0-c5e1-4912-adc2-80ea1f65f5fa service placement] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:05.762021 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220393a-fd8f-47fd-a18b-e60e62a7c00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:05.763050 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220393a-fd8f-47fd-a18b-e60e62a7c00d admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:05.763453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220393a-fd8f-47fd-a18b-e60e62a7c00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:05.763999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220393a-fd8f-47fd-a18b-e60e62a7c00d admin admin] PATH_INFO: `/v3/projects/2f05e6ff2db64134abbe5c742fe8652f/users/73ff8cb436b64e8ea800805e4aa2e377/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:05.787906 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97f8a5-f412-4ca3-b567-b5ed9151d823 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:05.790917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-cb97f8a5-f412-4ca3-b567-b5ed9151d823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:05.795776 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220393a-fd8f-47fd-a18b-e60e62a7c00d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f05e6ff2db64134abbe5c742fe8652f, user_id=73ff8cb436b64e8ea800805e4aa2e377, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:05.802577 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220393a-fd8f-47fd-a18b-e60e62a7c00d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:05.809611 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97f8a5-f412-4ca3-b567-b5ed9151d823 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:05.815596 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8a51d-a712-4c20-b253-4e722483453e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:05.816855 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8a51d-a712-4c20-b253-4e722483453e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:05.817242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8a51d-a712-4c20-b253-4e722483453e admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:05.817625 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8a51d-a712-4c20-b253-4e722483453e admin admin] PATH_INFO: `/v3/projects/ef2947427ce24f46baad0172db285cc8` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:05.819002 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8a51d-a712-4c20-b253-4e722483453e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef2947427ce24f46baad0172db285cc8)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:05.821398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8a51d-a712-4c20-b253-4e722483453e admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:05.894071 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1326/2641] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:54:05 2022] PUT /identity/v3/projects/2f05e6ff2db64134abbe5c742fe8652f/users/73ff8cb436b64e8ea800805e4aa2e377/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:05.905448 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dd559-7403-45dc-aea4-2796b3282bac None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:05.905669 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dd559-7403-45dc-aea4-2796b3282bac None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:05.906020 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dd559-7403-45dc-aea4-2796b3282bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:05.938805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2ce8a51d-a712-4c20-b253-4e722483453e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef2947427ce24f46baad0172db285cc8', 'request_id': 'req-2ce8a51d-a712-4c20-b253-4e722483453e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:05.941137 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-329dd559-7403-45dc-aea4-2796b3282bac None None] MFA Rules not processed for user `73ff8cb436b64e8ea800805e4aa2e377`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:05.942382 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-329dd559-7403-45dc-aea4-2796b3282bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:05.946137 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2ce8a51d-a712-4c20-b253-4e722483453e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef2947427ce24f46baad0172db285cc8', 'request_id': 'req-2ce8a51d-a712-4c20-b253-4e722483453e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:05.953826 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2ce8a51d-a712-4c20-b253-4e722483453e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef2947427ce24f46baad0172db285cc8', 'request_id': 'req-2ce8a51d-a712-4c20-b253-4e722483453e'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:06.012739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1327/2642] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:54:06.018144 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dd559-7403-45dc-aea4-2796b3282bac None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:06.080578 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2ce8a51d-a712-4c20-b253-4e722483453e admin admin] The token cache is being invalidate because project ef2947427ce24f46baad0172db285cc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:54:06.080991 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2ce8a51d-a712-4c20-b253-4e722483453e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:06.081818 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2ce8a51d-a712-4c20-b253-4e722483453e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:06.083644 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1316/2643] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:54:05 2022] DELETE /identity/v3/projects/ef2947427ce24f46baad0172db285cc8 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:06.088543 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb59233-3e6b-47d2-8a25-e772fca18978 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:06.088832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb59233-3e6b-47d2-8a25-e772fca18978 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:06.089113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb59233-3e6b-47d2-8a25-e772fca18978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:06.096429 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dd559-7403-45dc-aea4-2796b3282bac None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:06.101358 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82c4bf-1c44-45af-b452-fccc65e5cac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:06.102029 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82c4bf-1c44-45af-b452-fccc65e5cac3 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:06.102320 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82c4bf-1c44-45af-b452-fccc65e5cac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:06.102600 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82c4bf-1c44-45af-b452-fccc65e5cac3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:06.103308 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82c4bf-1c44-45af-b452-fccc65e5cac3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:06.105341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82c4bf-1c44-45af-b452-fccc65e5cac3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:06.108106 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0fb59233-3e6b-47d2-8a25-e772fca18978 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:06.109322 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0fb59233-3e6b-47d2-8a25-e772fca18978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:06.117660 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1317/2644] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:54:06.125451 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4989167-e6d3-4ae8-9f14-9f8c113a49fc None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:06.125723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4989167-e6d3-4ae8-9f14-9f8c113a49fc None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:06.126011 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4989167-e6d3-4ae8-9f14-9f8c113a49fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:06.145940 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a4989167-e6d3-4ae8-9f14-9f8c113a49fc None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:06.147260 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a4989167-e6d3-4ae8-9f14-9f8c113a49fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:06.156848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1318/2645] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:54:06.160987 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4989167-e6d3-4ae8-9f14-9f8c113a49fc None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:06.170973 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4989167-e6d3-4ae8-9f14-9f8c113a49fc None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:06.176594 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32857319-1c75-490f-a831-c73f5c42b6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:06.176837 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32857319-1c75-490f-a831-c73f5c42b6ad admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:06.177188 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32857319-1c75-490f-a831-c73f5c42b6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:06.177467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32857319-1c75-490f-a831-c73f5c42b6ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:06.178199 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32857319-1c75-490f-a831-c73f5c42b6ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:54:06.178500 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32857319-1c75-490f-a831-c73f5c42b6ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:06.180133 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32857319-1c75-490f-a831-c73f5c42b6ad admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:06.197313 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1319/2646] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:54:06.200826 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32857319-1c75-490f-a831-c73f5c42b6ad admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:06.208905 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32857319-1c75-490f-a831-c73f5c42b6ad admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:06.213914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47139bd-78d5-47bb-9f4e-7a28bd97a804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:06.214561 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47139bd-78d5-47bb-9f4e-7a28bd97a804 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:06.214843 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47139bd-78d5-47bb-9f4e-7a28bd97a804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:06.215116 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47139bd-78d5-47bb-9f4e-7a28bd97a804 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:06.216425 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47139bd-78d5-47bb-9f4e-7a28bd97a804 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:06.218997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47139bd-78d5-47bb-9f4e-7a28bd97a804 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:06.229145 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1328/2647] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:54:06 2022] POST /identity/v3/projects => generated 377 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:54:06.232864 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82c4bf-1c44-45af-b452-fccc65e5cac3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:06.235065 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ab82c4bf-1c44-45af-b452-fccc65e5cac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:06.305636 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82c4bf-1c44-45af-b452-fccc65e5cac3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:06.310642 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05286b3c-456f-4ecf-b4d8-b4ef2695861c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:06.311309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05286b3c-456f-4ecf-b4d8-b4ef2695861c admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:06.311592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05286b3c-456f-4ecf-b4d8-b4ef2695861c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:06.311898 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05286b3c-456f-4ecf-b4d8-b4ef2695861c admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:06.312584 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05286b3c-456f-4ecf-b4d8-b4ef2695861c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:06.315090 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05286b3c-456f-4ecf-b4d8-b4ef2695861c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:06.348148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1320/2648] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:54:06 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:54:06.352509 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47139bd-78d5-47bb-9f4e-7a28bd97a804 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:06.362950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:06.415541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47139bd-78d5-47bb-9f4e-7a28bd97a804 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:06.420083 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1329/2649] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:54:06 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:54:06.420308 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9df5f-cf41-4058-a433-d7f04814c139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:06.420967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9df5f-cf41-4058-a433-d7f04814c139 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:06.421254 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9df5f-cf41-4058-a433-d7f04814c139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:06.421528 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9df5f-cf41-4058-a433-d7f04814c139 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:06.422222 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9df5f-cf41-4058-a433-d7f04814c139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:06.423987 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05286b3c-456f-4ecf-b4d8-b4ef2695861c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:06.424159 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9df5f-cf41-4058-a433-d7f04814c139 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:06.434224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05286b3c-456f-4ecf-b4d8-b4ef2695861c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:06.439472 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f6005-0789-4d75-aef4-ad53af71b105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:06.440160 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f6005-0789-4d75-aef4-ad53af71b105 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:06.440444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f6005-0789-4d75-aef4-ad53af71b105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:06.440719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f6005-0789-4d75-aef4-ad53af71b105 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:06.441481 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f6005-0789-4d75-aef4-ad53af71b105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:06.443095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f6005-0789-4d75-aef4-ad53af71b105 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:06.453686 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1330/2650] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:54:06.457783 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f6005-0789-4d75-aef4-ad53af71b105 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:06.468517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f6005-0789-4d75-aef4-ad53af71b105 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:06.474421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cfb27-b360-4b0b-a2c7-791956fd561d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:06.475356 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cfb27-b360-4b0b-a2c7-791956fd561d admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:06.475945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cfb27-b360-4b0b-a2c7-791956fd561d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:06.476405 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cfb27-b360-4b0b-a2c7-791956fd561d admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:06.477481 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81cfb27-b360-4b0b-a2c7-791956fd561d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:06.480781 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81cfb27-b360-4b0b-a2c7-791956fd561d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:06.492865 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1331/2651] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:54:06.497272 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cfb27-b360-4b0b-a2c7-791956fd561d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:06.508610 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cfb27-b360-4b0b-a2c7-791956fd561d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:06.514751 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1412e-0bdb-4056-8757-82d8860354f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:06.515644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1412e-0bdb-4056-8757-82d8860354f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:06.516146 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1412e-0bdb-4056-8757-82d8860354f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:06.516288 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1412e-0bdb-4056-8757-82d8860354f6 admin admin] PATH_INFO: `/v3/projects/b58bda1980e4445685885fccdd532178/users/1186e9374b2144399b2202ded289cfbf/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:06.533574 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1412e-0bdb-4056-8757-82d8860354f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b58bda1980e4445685885fccdd532178, user_id=1186e9374b2144399b2202ded289cfbf, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:06.535442 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1321/2652] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:54:06 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:54:06.536385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1412e-0bdb-4056-8757-82d8860354f6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:06.539703 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9df5f-cf41-4058-a433-d7f04814c139 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:06.545056 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:06.550425 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9df5f-cf41-4058-a433-d7f04814c139 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:06.555168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b4d2a-59e2-4372-8dde-19aa213af6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:06.555865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b4d2a-59e2-4372-8dde-19aa213af6d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:06.556152 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b4d2a-59e2-4372-8dde-19aa213af6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:06.556427 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b4d2a-59e2-4372-8dde-19aa213af6d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:06.557154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b4d2a-59e2-4372-8dde-19aa213af6d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:06.558773 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b4d2a-59e2-4372-8dde-19aa213af6d1 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:06.569541 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1322/2653] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:54:06.573792 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b4d2a-59e2-4372-8dde-19aa213af6d1 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:06.585304 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b4d2a-59e2-4372-8dde-19aa213af6d1 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:06.589956 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a94bbc-0f49-4704-94e1-44ee10807f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:06.590605 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a94bbc-0f49-4704-94e1-44ee10807f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:06.591411 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a94bbc-0f49-4704-94e1-44ee10807f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:06.592302 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a94bbc-0f49-4704-94e1-44ee10807f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:06.593076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a94bbc-0f49-4704-94e1-44ee10807f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:06.594658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a94bbc-0f49-4704-94e1-44ee10807f8d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:06.605175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1323/2654] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:54:06.609300 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a94bbc-0f49-4704-94e1-44ee10807f8d admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:06.618736 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a94bbc-0f49-4704-94e1-44ee10807f8d admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:06.623909 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d49a0-e113-48dd-9a04-38374982f1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:06.624749 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d49a0-e113-48dd-9a04-38374982f1bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:06.625024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d49a0-e113-48dd-9a04-38374982f1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:06.625296 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d49a0-e113-48dd-9a04-38374982f1bc admin admin] PATH_INFO: `/v3/projects/690633edfd1847cf9ca6726401bfc53c/users/56a5a8e24609483da8553b88efe97940/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:06.633491 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1332/2655] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:54:06 2022] PUT /identity/v3/projects/b58bda1980e4445685885fccdd532178/users/1186e9374b2144399b2202ded289cfbf/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:06.646487 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d49a0-e113-48dd-9a04-38374982f1bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=690633edfd1847cf9ca6726401bfc53c, user_id=56a5a8e24609483da8553b88efe97940, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:06.649223 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856d49a0-e113-48dd-9a04-38374982f1bc admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:06.649376 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe2c69-394f-4bb8-b2c7-aa6fb2a53886 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:06.649728 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe2c69-394f-4bb8-b2c7-aa6fb2a53886 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:06.650029 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe2c69-394f-4bb8-b2c7-aa6fb2a53886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:06.678199 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e7fe2c69-394f-4bb8-b2c7-aa6fb2a53886 None None] MFA Rules not processed for user `1186e9374b2144399b2202ded289cfbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:06.679424 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e7fe2c69-394f-4bb8-b2c7-aa6fb2a53886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:06.721506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1324/2656] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:54:06 2022] PUT /identity/v3/projects/690633edfd1847cf9ca6726401bfc53c/users/56a5a8e24609483da8553b88efe97940/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:54:06.732874 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98829d3-77f0-4c5d-809f-6e8f82e31d36 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:06.733186 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98829d3-77f0-4c5d-809f-6e8f82e31d36 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:06.733441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98829d3-77f0-4c5d-809f-6e8f82e31d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:06.763756 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1333/2657] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:54:06.768946 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe2c69-394f-4bb8-b2c7-aa6fb2a53886 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:06.776931 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a98829d3-77f0-4c5d-809f-6e8f82e31d36 None None] MFA Rules not processed for user `56a5a8e24609483da8553b88efe97940`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:06.778823 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a98829d3-77f0-4c5d-809f-6e8f82e31d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:06.857488 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:06.862439 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe2c69-394f-4bb8-b2c7-aa6fb2a53886 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:06.871247 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1325/2658] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:06.883840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98829d3-77f0-4c5d-809f-6e8f82e31d36 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:06.886591 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a98829d3-77f0-4c5d-809f-6e8f82e31d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:06.948173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd097f-ae94-418c-9269-5c781c09af01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:06.949048 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd097f-ae94-418c-9269-5c781c09af01 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:06.949445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd097f-ae94-418c-9269-5c781c09af01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:06.949839 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd097f-ae94-418c-9269-5c781c09af01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:06.951400 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd097f-ae94-418c-9269-5c781c09af01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:06.954052 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd097f-ae94-418c-9269-5c781c09af01 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:07.037677 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98829d3-77f0-4c5d-809f-6e8f82e31d36 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:07.044684 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665ca02-cff0-4f2c-9e2c-24c20ee912bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:07.045422 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665ca02-cff0-4f2c-9e2c-24c20ee912bf service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:07.045720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665ca02-cff0-4f2c-9e2c-24c20ee912bf service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:07.045992 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665ca02-cff0-4f2c-9e2c-24c20ee912bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:07.054849 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665ca02-cff0-4f2c-9e2c-24c20ee912bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:07.056521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665ca02-cff0-4f2c-9e2c-24c20ee912bf service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:07.067639 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1326/2659] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:54:07.105783 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1334/2660] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:54:06 2022] POST /identity/v3/projects => generated 375 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:54:07.114135 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665ca02-cff0-4f2c-9e2c-24c20ee912bf service nova] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:07.125433 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:07.134384 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665ca02-cff0-4f2c-9e2c-24c20ee912bf service nova] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:07.221315 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d0165-a354-495e-b92a-a6fa27f0845b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:07.221898 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d0165-a354-495e-b92a-a6fa27f0845b admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:07.222184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d0165-a354-495e-b92a-a6fa27f0845b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:07.222461 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d0165-a354-495e-b92a-a6fa27f0845b admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:07.223442 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d0165-a354-495e-b92a-a6fa27f0845b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:07.226102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d0165-a354-495e-b92a-a6fa27f0845b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:07.421473 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1327/2661] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:54:07 2022] POST /identity/v3/users => generated 294 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:54:07.425638 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd097f-ae94-418c-9269-5c781c09af01 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:07.437009 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd097f-ae94-418c-9269-5c781c09af01 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:07.441933 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf9793-f1b3-43cd-8377-3cadec4c2e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:07.442609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf9793-f1b3-43cd-8377-3cadec4c2e4e admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:07.445887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf9793-f1b3-43cd-8377-3cadec4c2e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:07.445887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf9793-f1b3-43cd-8377-3cadec4c2e4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:07.445887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf9793-f1b3-43cd-8377-3cadec4c2e4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:07.445887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf9793-f1b3-43cd-8377-3cadec4c2e4e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:07.458961 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1335/2662] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:54:07.463422 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d0165-a354-495e-b92a-a6fa27f0845b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:07.474638 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d0165-a354-495e-b92a-a6fa27f0845b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:07.479847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab23bd-1a03-4d16-b74b-697f927cac76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:07.480850 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab23bd-1a03-4d16-b74b-697f927cac76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:07.481242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab23bd-1a03-4d16-b74b-697f927cac76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:07.482381 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab23bd-1a03-4d16-b74b-697f927cac76 admin admin] PATH_INFO: `/v3/projects/39ec7a9a14cd41aa97d4d4ddb16c21fc/users/0caf1a659f074d089aba31bddee44e15/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:07.499454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab23bd-1a03-4d16-b74b-697f927cac76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ec7a9a14cd41aa97d4d4ddb16c21fc, user_id=0caf1a659f074d089aba31bddee44e15, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:07.502283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab23bd-1a03-4d16-b74b-697f927cac76 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:07.596249 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1328/2663] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:54:07 2022] PUT /identity/v3/projects/39ec7a9a14cd41aa97d4d4ddb16c21fc/users/0caf1a659f074d089aba31bddee44e15/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:07.597603 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf9793-f1b3-43cd-8377-3cadec4c2e4e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:07.608592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf9793-f1b3-43cd-8377-3cadec4c2e4e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:07.681580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63675c8f-cf99-4c1d-b62f-8b09113c1d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:07.682675 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63675c8f-cf99-4c1d-b62f-8b09113c1d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:07.683114 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63675c8f-cf99-4c1d-b62f-8b09113c1d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:07.683453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63675c8f-cf99-4c1d-b62f-8b09113c1d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:07.684454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63675c8f-cf99-4c1d-b62f-8b09113c1d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:07.686823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63675c8f-cf99-4c1d-b62f-8b09113c1d72 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:07.699186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1336/2664] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:54:07.704572 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab23bd-1a03-4d16-b74b-697f927cac76 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:07.715166 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab23bd-1a03-4d16-b74b-697f927cac76 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:07.720745 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2ad3f-a024-4c79-93bb-b28d13d9dade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:07.721630 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2ad3f-a024-4c79-93bb-b28d13d9dade admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:07.721907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2ad3f-a024-4c79-93bb-b28d13d9dade admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:07.722182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2ad3f-a024-4c79-93bb-b28d13d9dade admin admin] PATH_INFO: `/v3/projects/39ec7a9a14cd41aa97d4d4ddb16c21fc/users/0caf1a659f074d089aba31bddee44e15/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:07.724261 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2ad3f-a024-4c79-93bb-b28d13d9dade admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ec7a9a14cd41aa97d4d4ddb16c21fc, user_id=0caf1a659f074d089aba31bddee44e15, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:07.726962 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2ad3f-a024-4c79-93bb-b28d13d9dade admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:07.739525 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1329/2665] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:54:07 2022] PUT /identity/v3/projects/39ec7a9a14cd41aa97d4d4ddb16c21fc/users/0caf1a659f074d089aba31bddee44e15/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:07.749943 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0179b-37d4-4dc7-97aa-19f870d2745e None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:07.751255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0179b-37d4-4dc7-97aa-19f870d2745e None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:07.751255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0179b-37d4-4dc7-97aa-19f870d2745e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:07.781170 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8cf0179b-37d4-4dc7-97aa-19f870d2745e None None] MFA Rules not processed for user `0caf1a659f074d089aba31bddee44e15`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:07.782597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8cf0179b-37d4-4dc7-97aa-19f870d2745e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:07.850188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:07.853905 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1337/2666] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:07 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:07.865471 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2ad3f-a024-4c79-93bb-b28d13d9dade admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:07.867283 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-71a2ad3f-a024-4c79-93bb-b28d13d9dade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:07.928692 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2ad3f-a024-4c79-93bb-b28d13d9dade admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:07.932369 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b708e-264a-4919-9608-27e03a470ff5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:07.933010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b708e-264a-4919-9608-27e03a470ff5 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:07.933291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b708e-264a-4919-9608-27e03a470ff5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:07.933566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b708e-264a-4919-9608-27e03a470ff5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:07.936005 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-734b708e-264a-4919-9608-27e03a470ff5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:08.011402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b708e-264a-4919-9608-27e03a470ff5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:08.013567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b708e-264a-4919-9608-27e03a470ff5 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:08.024904 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1330/2667] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:54:10.192871 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0179b-37d4-4dc7-97aa-19f870d2745e None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:10.203911 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0179b-37d4-4dc7-97aa-19f870d2745e None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:10.207578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee49b8d-849b-489b-bfc2-e034c71610ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:10.208342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee49b8d-849b-489b-bfc2-e034c71610ac service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:10.213499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee49b8d-849b-489b-bfc2-e034c71610ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:10.213499 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee49b8d-849b-489b-bfc2-e034c71610ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:10.269826 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee49b8d-849b-489b-bfc2-e034c71610ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:10.271289 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee49b8d-849b-489b-bfc2-e034c71610ac service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:10.281287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1338/2668] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:54:12.394064 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b708e-264a-4919-9608-27e03a470ff5 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:12.402817 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b708e-264a-4919-9608-27e03a470ff5 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:12.407103 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef13975-a8ad-4450-a7f3-a02f511022e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:12.407767 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef13975-a8ad-4450-a7f3-a02f511022e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:12.408061 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef13975-a8ad-4450-a7f3-a02f511022e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:12.408343 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef13975-a8ad-4450-a7f3-a02f511022e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:12.417364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef13975-a8ad-4450-a7f3-a02f511022e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:12.418821 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef13975-a8ad-4450-a7f3-a02f511022e4 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:12.422932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:12.429271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1331/2669] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:12 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:54:15.808850 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee49b8d-849b-489b-bfc2-e034c71610ac service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:15.906131 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee49b8d-849b-489b-bfc2-e034c71610ac service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:15.911755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5569f5-c76f-46dc-8977-df3f070b7e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:15.912782 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5569f5-c76f-46dc-8977-df3f070b7e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:15.913073 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5569f5-c76f-46dc-8977-df3f070b7e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:15.913349 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5569f5-c76f-46dc-8977-df3f070b7e98 admin admin] PATH_INFO: `/v3/users/56a5a8e24609483da8553b88efe97940` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:15.915308 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5569f5-c76f-46dc-8977-df3f070b7e98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56a5a8e24609483da8553b88efe97940)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:15.917505 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5569f5-c76f-46dc-8977-df3f070b7e98 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:16.237195 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3e5569f5-c76f-46dc-8977-df3f070b7e98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56a5a8e24609483da8553b88efe97940'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:16.238044 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3e5569f5-c76f-46dc-8977-df3f070b7e98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56a5a8e24609483da8553b88efe97940'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:16.238662 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3e5569f5-c76f-46dc-8977-df3f070b7e98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56a5a8e24609483da8553b88efe97940'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:16.264064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3e5569f5-c76f-46dc-8977-df3f070b7e98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56a5a8e24609483da8553b88efe97940'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:16.273485 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3e5569f5-c76f-46dc-8977-df3f070b7e98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56a5a8e24609483da8553b88efe97940'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:16.386689 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1339/2670] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:54:15 2022] DELETE /identity/v3/users/56a5a8e24609483da8553b88efe97940 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:16.392203 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1879df-631c-4eca-af00-6e67941364cf None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:16.392750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1879df-631c-4eca-af00-6e67941364cf None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:16.392750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1879df-631c-4eca-af00-6e67941364cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:16.411546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-bc1879df-631c-4eca-af00-6e67941364cf None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:16.412801 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bc1879df-631c-4eca-af00-6e67941364cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:16.487495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1332/2671] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:16.499082 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5569f5-c76f-46dc-8977-df3f070b7e98 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:16.500933 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-3e5569f5-c76f-46dc-8977-df3f070b7e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:16.524990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:16.567651 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5569f5-c76f-46dc-8977-df3f070b7e98 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:16.571413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb69419-b0b3-4795-80a7-f25ba10a5f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:16.572183 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb69419-b0b3-4795-80a7-f25ba10a5f90 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:16.572477 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb69419-b0b3-4795-80a7-f25ba10a5f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:16.572750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb69419-b0b3-4795-80a7-f25ba10a5f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:16.584720 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb69419-b0b3-4795-80a7-f25ba10a5f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:16.586178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb69419-b0b3-4795-80a7-f25ba10a5f90 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:16.597313 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1340/2672] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:54:17.155073 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1879df-631c-4eca-af00-6e67941364cf None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:17.156378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bc1879df-631c-4eca-af00-6e67941364cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:17.169043 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1879df-631c-4eca-af00-6e67941364cf None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:17.177838 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be49882-13eb-45e7-b9f0-72c859f2bbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:17.178932 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be49882-13eb-45e7-b9f0-72c859f2bbff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:17.179249 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be49882-13eb-45e7-b9f0-72c859f2bbff admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:17.179577 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be49882-13eb-45e7-b9f0-72c859f2bbff admin admin] PATH_INFO: `/v3/projects/690633edfd1847cf9ca6726401bfc53c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:17.181026 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be49882-13eb-45e7-b9f0-72c859f2bbff admin admin] RBAC: Authorizing `identity:delete_project(project_id=690633edfd1847cf9ca6726401bfc53c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:17.183750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be49882-13eb-45e7-b9f0-72c859f2bbff admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:17.336685 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4be49882-13eb-45e7-b9f0-72c859f2bbff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '690633edfd1847cf9ca6726401bfc53c', 'request_id': 'req-4be49882-13eb-45e7-b9f0-72c859f2bbff'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:17.346382 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4be49882-13eb-45e7-b9f0-72c859f2bbff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '690633edfd1847cf9ca6726401bfc53c', 'request_id': 'req-4be49882-13eb-45e7-b9f0-72c859f2bbff'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:17.355959 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4be49882-13eb-45e7-b9f0-72c859f2bbff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '690633edfd1847cf9ca6726401bfc53c', 'request_id': 'req-4be49882-13eb-45e7-b9f0-72c859f2bbff'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:17.404611 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb69419-b0b3-4795-80a7-f25ba10a5f90 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:17.406199 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cdb69419-b0b3-4795-80a7-f25ba10a5f90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:17.419400 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb69419-b0b3-4795-80a7-f25ba10a5f90 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:17.457903 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4be49882-13eb-45e7-b9f0-72c859f2bbff admin admin] The token cache is being invalidate because project 690633edfd1847cf9ca6726401bfc53c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:54:17.459217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4be49882-13eb-45e7-b9f0-72c859f2bbff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:17.460223 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-4be49882-13eb-45e7-b9f0-72c859f2bbff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:17.462763 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1333/2673] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:54:17 2022] DELETE /identity/v3/projects/690633edfd1847cf9ca6726401bfc53c => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:17.468756 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ef189-b697-4ac8-b5b1-f7c93c39ec94 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:17.469074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ef189-b697-4ac8-b5b1-f7c93c39ec94 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:17.469335 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ef189-b697-4ac8-b5b1-f7c93c39ec94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:17.493431 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-531ef189-b697-4ac8-b5b1-f7c93c39ec94 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:17.499172 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-531ef189-b697-4ac8-b5b1-f7c93c39ec94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:17.499359 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903dd85-514b-448b-8e9c-e4f2038a884d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:17.499359 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903dd85-514b-448b-8e9c-e4f2038a884d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:17.499359 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903dd85-514b-448b-8e9c-e4f2038a884d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:17.499359 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903dd85-514b-448b-8e9c-e4f2038a884d admin admin] PATH_INFO: `/v3/users/73ff8cb436b64e8ea800805e4aa2e377` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:17.500931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903dd85-514b-448b-8e9c-e4f2038a884d admin admin] RBAC: Authorizing `identity:delete_user(user_id=73ff8cb436b64e8ea800805e4aa2e377)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:17.502907 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903dd85-514b-448b-8e9c-e4f2038a884d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:17.505190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1334/2674] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:17.514712 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea7add-9b93-45d1-8886-71b0357590ea None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:17.515151 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea7add-9b93-45d1-8886-71b0357590ea None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:17.515517 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea7add-9b93-45d1-8886-71b0357590ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:17.541327 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-c2ea7add-9b93-45d1-8886-71b0357590ea None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:17.542929 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-c2ea7add-9b93-45d1-8886-71b0357590ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:17.552135 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1335/2675] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:17.562383 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea7add-9b93-45d1-8886-71b0357590ea None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:17.569623 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea7add-9b93-45d1-8886-71b0357590ea None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:17.578320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa475b0-173f-4c97-83cf-bae1e8dc33bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:17.579106 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa475b0-173f-4c97-83cf-bae1e8dc33bd admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:17.579390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa475b0-173f-4c97-83cf-bae1e8dc33bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:17.579865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa475b0-173f-4c97-83cf-bae1e8dc33bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:17.580554 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa475b0-173f-4c97-83cf-bae1e8dc33bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:54:17.580885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa475b0-173f-4c97-83cf-bae1e8dc33bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:17.583568 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa475b0-173f-4c97-83cf-bae1e8dc33bd admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:17.604557 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1336/2676] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:54:17.608193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa475b0-173f-4c97-83cf-bae1e8dc33bd admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:17.619055 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa475b0-173f-4c97-83cf-bae1e8dc33bd admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:17.624711 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c4d3e5-96d4-4c43-abca-ea94b06c03cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:17.625398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4d3e5-96d4-4c43-abca-ea94b06c03cb admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:17.625756 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4d3e5-96d4-4c43-abca-ea94b06c03cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:17.625975 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4d3e5-96d4-4c43-abca-ea94b06c03cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:17.626653 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c4d3e5-96d4-4c43-abca-ea94b06c03cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:17.628701 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c4d3e5-96d4-4c43-abca-ea94b06c03cb admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:17.807374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1337/2677] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:54:17 2022] POST /identity/v3/projects => generated 381 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:54:17.811949 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c4d3e5-96d4-4c43-abca-ea94b06c03cb admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:17.853938 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:17.883920 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9903dd85-514b-448b-8e9c-e4f2038a884d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73ff8cb436b64e8ea800805e4aa2e377'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:17.884680 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9903dd85-514b-448b-8e9c-e4f2038a884d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73ff8cb436b64e8ea800805e4aa2e377'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:17.885329 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9903dd85-514b-448b-8e9c-e4f2038a884d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73ff8cb436b64e8ea800805e4aa2e377'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:17.893924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c4d3e5-96d4-4c43-abca-ea94b06c03cb admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:17.902411 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1cb324-715e-473a-ab6a-44ada99176a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:17.903453 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cb324-715e-473a-ab6a-44ada99176a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:17.903777 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cb324-715e-473a-ab6a-44ada99176a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:17.904274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cb324-715e-473a-ab6a-44ada99176a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:17.909668 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1cb324-715e-473a-ab6a-44ada99176a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:17.909668 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1cb324-715e-473a-ab6a-44ada99176a3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:17.909956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9903dd85-514b-448b-8e9c-e4f2038a884d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73ff8cb436b64e8ea800805e4aa2e377'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:17.916983 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9903dd85-514b-448b-8e9c-e4f2038a884d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73ff8cb436b64e8ea800805e4aa2e377'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:17.998252 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1338/2678] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:54:17 2022] POST /identity/v3/users => generated 291 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:54:18.002213 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1cb324-715e-473a-ab6a-44ada99176a3 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:18.004166 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-dc1cb324-715e-473a-ab6a-44ada99176a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:18.009581 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1341/2679] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:54:17 2022] DELETE /identity/v3/users/73ff8cb436b64e8ea800805e4aa2e377 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:18.015532 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d1846-ccb3-47da-ba9b-ab1f0de04672 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:18.015835 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d1846-ccb3-47da-ba9b-ab1f0de04672 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:18.016167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d1846-ccb3-47da-ba9b-ab1f0de04672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:18.017640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1cb324-715e-473a-ab6a-44ada99176a3 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:18.037274 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-ec6d1846-ccb3-47da-ba9b-ab1f0de04672 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:18.038512 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ec6d1846-ccb3-47da-ba9b-ab1f0de04672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:18.102919 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:18.104168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9ee3a-aee5-46af-99bd-21b981bf6d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:18.104869 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9ee3a-aee5-46af-99bd-21b981bf6d6b admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:18.105166 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9ee3a-aee5-46af-99bd-21b981bf6d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:18.105455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9ee3a-aee5-46af-99bd-21b981bf6d6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:18.106186 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9ee3a-aee5-46af-99bd-21b981bf6d6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:18.108946 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9ee3a-aee5-46af-99bd-21b981bf6d6b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:18.130403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1339/2680] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:54:18.131627 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9ee3a-aee5-46af-99bd-21b981bf6d6b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:18.146148 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1342/2681] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:18.146356 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9ee3a-aee5-46af-99bd-21b981bf6d6b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:18.154813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc162a1-f3db-4fd5-bd2b-d1e1e153d32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:18.156028 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc162a1-f3db-4fd5-bd2b-d1e1e153d32b admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:18.156542 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc162a1-f3db-4fd5-bd2b-d1e1e153d32b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:18.157228 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc162a1-f3db-4fd5-bd2b-d1e1e153d32b admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:18.158695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d1846-ccb3-47da-ba9b-ab1f0de04672 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:18.159087 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc162a1-f3db-4fd5-bd2b-d1e1e153d32b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:18.160670 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ec6d1846-ccb3-47da-ba9b-ab1f0de04672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:18.162262 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc162a1-f3db-4fd5-bd2b-d1e1e153d32b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:18.183489 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1340/2682] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:54:18.188506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc162a1-f3db-4fd5-bd2b-d1e1e153d32b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:18.206818 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc162a1-f3db-4fd5-bd2b-d1e1e153d32b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:18.212326 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2096fef-8ce0-4627-8bcf-3998ec2e3a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:18.213341 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2096fef-8ce0-4627-8bcf-3998ec2e3a28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:18.213626 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2096fef-8ce0-4627-8bcf-3998ec2e3a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:18.213902 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2096fef-8ce0-4627-8bcf-3998ec2e3a28 admin admin] PATH_INFO: `/v3/projects/636d96e7383a4dbd9d8c5f30c1c31b0c/users/955812fb4b284187ba9e6d0d3e165d0c/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:18.252216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2096fef-8ce0-4627-8bcf-3998ec2e3a28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=636d96e7383a4dbd9d8c5f30c1c31b0c, user_id=955812fb4b284187ba9e6d0d3e165d0c, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:18.259047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2096fef-8ce0-4627-8bcf-3998ec2e3a28 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:18.261472 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d1846-ccb3-47da-ba9b-ab1f0de04672 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:18.268340 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22a152-eb0b-4d91-9113-d3a2c4815f20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:18.268606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22a152-eb0b-4d91-9113-d3a2c4815f20 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:18.268722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22a152-eb0b-4d91-9113-d3a2c4815f20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:18.269021 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22a152-eb0b-4d91-9113-d3a2c4815f20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:18.280913 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c22a152-eb0b-4d91-9113-d3a2c4815f20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:18.282455 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c22a152-eb0b-4d91-9113-d3a2c4815f20 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:18.294438 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1343/2683] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:54:18.400928 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1341/2684] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:54:18 2022] PUT /identity/v3/projects/636d96e7383a4dbd9d8c5f30c1c31b0c/users/955812fb4b284187ba9e6d0d3e165d0c/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:54:18.412540 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e516e5-7dad-4fbc-a964-32409622ae6a None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:18.413137 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e516e5-7dad-4fbc-a964-32409622ae6a None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:18.413137 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e516e5-7dad-4fbc-a964-32409622ae6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:18.444737 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-57e516e5-7dad-4fbc-a964-32409622ae6a None None] MFA Rules not processed for user `955812fb4b284187ba9e6d0d3e165d0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:18.445975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-57e516e5-7dad-4fbc-a964-32409622ae6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:18.521687 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1344/2685] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:18.526564 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2096fef-8ce0-4627-8bcf-3998ec2e3a28 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:18.604815 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2096fef-8ce0-4627-8bcf-3998ec2e3a28 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:18.610595 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df795bf4-ed62-4e57-ada2-b35fc73baa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:18.611433 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df795bf4-ed62-4e57-ada2-b35fc73baa0d admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:18.611775 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df795bf4-ed62-4e57-ada2-b35fc73baa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:18.612029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df795bf4-ed62-4e57-ada2-b35fc73baa0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:18.613161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df795bf4-ed62-4e57-ada2-b35fc73baa0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:18.614860 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df795bf4-ed62-4e57-ada2-b35fc73baa0d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:18.784920 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1342/2686] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:54:18 2022] POST /identity/v3/projects => generated 385 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:54:18.789085 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e516e5-7dad-4fbc-a964-32409622ae6a None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:18.800467 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e516e5-7dad-4fbc-a964-32409622ae6a None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:18.816672 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df795bf4-ed62-4e57-ada2-b35fc73baa0d admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:18.818405 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-df795bf4-ed62-4e57-ada2-b35fc73baa0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:18.830182 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:18.886464 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536c1d5-d82f-483f-a96f-fa9dffeb900b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:18.892121 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536c1d5-d82f-483f-a96f-fa9dffeb900b admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:18.892121 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536c1d5-d82f-483f-a96f-fa9dffeb900b admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:18.892121 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536c1d5-d82f-483f-a96f-fa9dffeb900b admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:18.892121 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536c1d5-d82f-483f-a96f-fa9dffeb900b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:18.892121 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536c1d5-d82f-483f-a96f-fa9dffeb900b admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:18.924894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df795bf4-ed62-4e57-ada2-b35fc73baa0d admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:18.930213 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d0232-4704-4531-8739-3ad0dd520808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:18.931115 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d0232-4704-4531-8739-3ad0dd520808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:18.931397 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d0232-4704-4531-8739-3ad0dd520808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:18.931703 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d0232-4704-4531-8739-3ad0dd520808 admin admin] PATH_INFO: `/v3/projects/2f05e6ff2db64134abbe5c742fe8652f` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:18.933181 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d0232-4704-4531-8739-3ad0dd520808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f05e6ff2db64134abbe5c742fe8652f)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:18.935652 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d0232-4704-4531-8739-3ad0dd520808 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:18.978030 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1345/2687] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:54:18 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:54:18.982990 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536c1d5-d82f-483f-a96f-fa9dffeb900b admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:18.993923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536c1d5-d82f-483f-a96f-fa9dffeb900b admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:18.998332 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:19.002294 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3863ef-b2a9-4c11-8eaf-6152ede7e78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:19.003991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3863ef-b2a9-4c11-8eaf-6152ede7e78f admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:19.003991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3863ef-b2a9-4c11-8eaf-6152ede7e78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:19.003991 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3863ef-b2a9-4c11-8eaf-6152ede7e78f admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:19.004403 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3863ef-b2a9-4c11-8eaf-6152ede7e78f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:19.006053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3863ef-b2a9-4c11-8eaf-6152ede7e78f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:19.018461 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1346/2688] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:54:19.022784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3863ef-b2a9-4c11-8eaf-6152ede7e78f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:19.035486 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3863ef-b2a9-4c11-8eaf-6152ede7e78f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:19.051536 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1b3d0232-4704-4531-8739-3ad0dd520808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f05e6ff2db64134abbe5c742fe8652f', 'request_id': 'req-1b3d0232-4704-4531-8739-3ad0dd520808'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:19.059343 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1b3d0232-4704-4531-8739-3ad0dd520808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f05e6ff2db64134abbe5c742fe8652f', 'request_id': 'req-1b3d0232-4704-4531-8739-3ad0dd520808'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:19.066790 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1b3d0232-4704-4531-8739-3ad0dd520808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f05e6ff2db64134abbe5c742fe8652f', 'request_id': 'req-1b3d0232-4704-4531-8739-3ad0dd520808'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:19.100463 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5eb131-8d2a-413d-a8b5-e2783c4ad3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:19.101134 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5eb131-8d2a-413d-a8b5-e2783c4ad3c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:19.101421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5eb131-8d2a-413d-a8b5-e2783c4ad3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:19.101700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5eb131-8d2a-413d-a8b5-e2783c4ad3c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:19.102435 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5eb131-8d2a-413d-a8b5-e2783c4ad3c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:19.104121 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5eb131-8d2a-413d-a8b5-e2783c4ad3c2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:19.114196 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1347/2689] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:54:19.118520 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5eb131-8d2a-413d-a8b5-e2783c4ad3c2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:19.128729 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5eb131-8d2a-413d-a8b5-e2783c4ad3c2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:19.135581 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747330d0-28d2-45f6-a25f-383f7af26bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:19.135581 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747330d0-28d2-45f6-a25f-383f7af26bb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:19.135581 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747330d0-28d2-45f6-a25f-383f7af26bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:19.135581 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747330d0-28d2-45f6-a25f-383f7af26bb7 admin admin] PATH_INFO: `/v3/projects/0f392356c22b44798ceb3725d62656bb/users/0cf300eecafd42c4888671f9872010f4/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:19.162038 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747330d0-28d2-45f6-a25f-383f7af26bb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f392356c22b44798ceb3725d62656bb, user_id=0cf300eecafd42c4888671f9872010f4, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:19.165245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747330d0-28d2-45f6-a25f-383f7af26bb7 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:19.166249 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1b3d0232-4704-4531-8739-3ad0dd520808 admin admin] The token cache is being invalidate because project 2f05e6ff2db64134abbe5c742fe8652f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:54:19.166549 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1b3d0232-4704-4531-8739-3ad0dd520808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:19.167242 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-1b3d0232-4704-4531-8739-3ad0dd520808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:19.169021 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1343/2690] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:54:18 2022] DELETE /identity/v3/projects/2f05e6ff2db64134abbe5c742fe8652f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:54:19.173281 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d0232-4704-4531-8739-3ad0dd520808 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:19.175079 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1b3d0232-4704-4531-8739-3ad0dd520808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:19.189390 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d0232-4704-4531-8739-3ad0dd520808 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:19.194908 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a8cd0-d1e1-4f95-9058-0eaf39a910af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:19.195847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a8cd0-d1e1-4f95-9058-0eaf39a910af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:19.196136 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a8cd0-d1e1-4f95-9058-0eaf39a910af admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:19.196414 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a8cd0-d1e1-4f95-9058-0eaf39a910af admin admin] PATH_INFO: `/v3/users/1186e9374b2144399b2202ded289cfbf` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:19.197777 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a8cd0-d1e1-4f95-9058-0eaf39a910af admin admin] RBAC: Authorizing `identity:delete_user(user_id=1186e9374b2144399b2202ded289cfbf)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:19.199974 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a8cd0-d1e1-4f95-9058-0eaf39a910af admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:19.270795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1348/2691] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:54:19 2022] PUT /identity/v3/projects/0f392356c22b44798ceb3725d62656bb/users/0cf300eecafd42c4888671f9872010f4/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:54:19.292030 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979d023-3ae2-43ae-8d81-6e95d096e779 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:19.292745 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979d023-3ae2-43ae-8d81-6e95d096e779 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:19.293027 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979d023-3ae2-43ae-8d81-6e95d096e779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:19.326710 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-9979d023-3ae2-43ae-8d81-6e95d096e779 None None] MFA Rules not processed for user `0cf300eecafd42c4888671f9872010f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:19.328262 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9979d023-3ae2-43ae-8d81-6e95d096e779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:19.408379 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1349/2692] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:54:19.413057 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979d023-3ae2-43ae-8d81-6e95d096e779 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:19.414704 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9979d023-3ae2-43ae-8d81-6e95d096e779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:19.446575 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-eb4a8cd0-d1e1-4f95-9058-0eaf39a910af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1186e9374b2144399b2202ded289cfbf'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:19.448661 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-eb4a8cd0-d1e1-4f95-9058-0eaf39a910af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1186e9374b2144399b2202ded289cfbf'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:19.449408 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-eb4a8cd0-d1e1-4f95-9058-0eaf39a910af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1186e9374b2144399b2202ded289cfbf'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:19.465949 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-eb4a8cd0-d1e1-4f95-9058-0eaf39a910af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1186e9374b2144399b2202ded289cfbf'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:19.477927 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-eb4a8cd0-d1e1-4f95-9058-0eaf39a910af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1186e9374b2144399b2202ded289cfbf'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:19.501635 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979d023-3ae2-43ae-8d81-6e95d096e779 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:19.508819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf63c8-9d23-4d63-9982-5bcbff3d0ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:19.509904 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf63c8-9d23-4d63-9982-5bcbff3d0ce7 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:19.510411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf63c8-9d23-4d63-9982-5bcbff3d0ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:19.510910 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf63c8-9d23-4d63-9982-5bcbff3d0ce7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:19.512789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf63c8-9d23-4d63-9982-5bcbff3d0ce7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:19.516555 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf63c8-9d23-4d63-9982-5bcbff3d0ce7 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:19.558236 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1344/2693] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:54:19 2022] DELETE /identity/v3/users/1186e9374b2144399b2202ded289cfbf => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:54:19.580489 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:19.684965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1350/2694] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:54:19 2022] POST /identity/v3/projects => generated 385 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:54:19.689506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a8cd0-d1e1-4f95-9058-0eaf39a910af admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:19.691037 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-eb4a8cd0-d1e1-4f95-9058-0eaf39a910af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:19.706884 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:19.768703 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a8cd0-d1e1-4f95-9058-0eaf39a910af admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:19.841278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d7e8f-2320-4051-9bc1-2d0a6e650a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:19.842570 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d7e8f-2320-4051-9bc1-2d0a6e650a84 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:19.842969 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d7e8f-2320-4051-9bc1-2d0a6e650a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:19.843143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d7e8f-2320-4051-9bc1-2d0a6e650a84 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:19.843873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d7e8f-2320-4051-9bc1-2d0a6e650a84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:19.845847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d7e8f-2320-4051-9bc1-2d0a6e650a84 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:19.953274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1345/2695] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:54:19 2022] POST /identity/v3/users => generated 299 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:54:19.959478 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf63c8-9d23-4d63-9982-5bcbff3d0ce7 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:19.971882 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf63c8-9d23-4d63-9982-5bcbff3d0ce7 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:19.976773 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12361e3-7a0e-4975-9db5-f5be3dd86e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:19.977747 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12361e3-7a0e-4975-9db5-f5be3dd86e84 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:19.977747 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12361e3-7a0e-4975-9db5-f5be3dd86e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:19.978062 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12361e3-7a0e-4975-9db5-f5be3dd86e84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:19.978724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12361e3-7a0e-4975-9db5-f5be3dd86e84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:19.980479 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12361e3-7a0e-4975-9db5-f5be3dd86e84 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:19.997876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1351/2696] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:54:20.004677 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d7e8f-2320-4051-9bc1-2d0a6e650a84 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:20.021865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d7e8f-2320-4051-9bc1-2d0a6e650a84 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:20.028462 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24970038-3ea2-4233-9b2b-00e07169eb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:20.030350 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24970038-3ea2-4233-9b2b-00e07169eb33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:20.030628 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24970038-3ea2-4233-9b2b-00e07169eb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:20.030911 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24970038-3ea2-4233-9b2b-00e07169eb33 admin admin] PATH_INFO: `/v3/projects/5d620400aff64ba28fd099f288900f64/users/0ea6e3ba3f384e27aad3fda7b50cf743/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:20.047483 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12361e3-7a0e-4975-9db5-f5be3dd86e84 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:20.050158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b12361e3-7a0e-4975-9db5-f5be3dd86e84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:20.055651 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24970038-3ea2-4233-9b2b-00e07169eb33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d620400aff64ba28fd099f288900f64, user_id=0ea6e3ba3f384e27aad3fda7b50cf743, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:20.061892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24970038-3ea2-4233-9b2b-00e07169eb33 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:20.077337 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12361e3-7a0e-4975-9db5-f5be3dd86e84 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:20.082672 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdada6f6-4e41-450f-a3cf-a0d70aa16850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:20.083589 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdada6f6-4e41-450f-a3cf-a0d70aa16850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:20.084240 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdada6f6-4e41-450f-a3cf-a0d70aa16850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:20.084529 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdada6f6-4e41-450f-a3cf-a0d70aa16850 admin admin] PATH_INFO: `/v3/projects/b58bda1980e4445685885fccdd532178` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:20.085880 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdada6f6-4e41-450f-a3cf-a0d70aa16850 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b58bda1980e4445685885fccdd532178)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:20.087974 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdada6f6-4e41-450f-a3cf-a0d70aa16850 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:20.147212 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1346/2697] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:54:20 2022] PUT /identity/v3/projects/5d620400aff64ba28fd099f288900f64/users/0ea6e3ba3f384e27aad3fda7b50cf743/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:20.151557 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24970038-3ea2-4233-9b2b-00e07169eb33 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:20.225615 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24970038-3ea2-4233-9b2b-00e07169eb33 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:20.228138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fdada6f6-4e41-450f-a3cf-a0d70aa16850 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b58bda1980e4445685885fccdd532178', 'request_id': 'req-fdada6f6-4e41-450f-a3cf-a0d70aa16850'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:20.240784 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fdada6f6-4e41-450f-a3cf-a0d70aa16850 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b58bda1980e4445685885fccdd532178', 'request_id': 'req-fdada6f6-4e41-450f-a3cf-a0d70aa16850'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:20.251247 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fdada6f6-4e41-450f-a3cf-a0d70aa16850 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b58bda1980e4445685885fccdd532178', 'request_id': 'req-fdada6f6-4e41-450f-a3cf-a0d70aa16850'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:20.262786 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:20.296534 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46232cfd-e425-4f2b-9585-0bc89df85df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:20.297227 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46232cfd-e425-4f2b-9585-0bc89df85df8 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:20.297518 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46232cfd-e425-4f2b-9585-0bc89df85df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:20.297791 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46232cfd-e425-4f2b-9585-0bc89df85df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:20.298654 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46232cfd-e425-4f2b-9585-0bc89df85df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:20.300327 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46232cfd-e425-4f2b-9585-0bc89df85df8 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:20.313017 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1347/2698] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:54:20.317427 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46232cfd-e425-4f2b-9585-0bc89df85df8 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:20.328704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46232cfd-e425-4f2b-9585-0bc89df85df8 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:20.333778 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73856dd7-9808-4960-98d1-0c12f7a9a1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:20.335894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73856dd7-9808-4960-98d1-0c12f7a9a1bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:20.335894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73856dd7-9808-4960-98d1-0c12f7a9a1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:20.335894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73856dd7-9808-4960-98d1-0c12f7a9a1bf admin admin] PATH_INFO: `/v3/projects/5d620400aff64ba28fd099f288900f64/users/0ea6e3ba3f384e27aad3fda7b50cf743/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:20.337365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73856dd7-9808-4960-98d1-0c12f7a9a1bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d620400aff64ba28fd099f288900f64, user_id=0ea6e3ba3f384e27aad3fda7b50cf743, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:20.340108 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73856dd7-9808-4960-98d1-0c12f7a9a1bf admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:20.348751 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fdada6f6-4e41-450f-a3cf-a0d70aa16850 admin admin] The token cache is being invalidate because project b58bda1980e4445685885fccdd532178 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:54:20.349104 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fdada6f6-4e41-450f-a3cf-a0d70aa16850 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:20.349829 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-fdada6f6-4e41-450f-a3cf-a0d70aa16850 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:20.351484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1352/2699] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:54:20 2022] DELETE /identity/v3/projects/b58bda1980e4445685885fccdd532178 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:54:20.355000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1348/2700] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:54:20 2022] PUT /identity/v3/projects/5d620400aff64ba28fd099f288900f64/users/0ea6e3ba3f384e27aad3fda7b50cf743/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:54:20.357903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdada6f6-4e41-450f-a3cf-a0d70aa16850 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:20.358984 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fdada6f6-4e41-450f-a3cf-a0d70aa16850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:20.366866 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1b056-95c4-4f7c-a666-2df76f763201 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:20.367312 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1b056-95c4-4f7c-a666-2df76f763201 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:20.367797 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1b056-95c4-4f7c-a666-2df76f763201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:20.378753 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdada6f6-4e41-450f-a3cf-a0d70aa16850 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:20.408500 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-bcd1b056-95c4-4f7c-a666-2df76f763201 None None] MFA Rules not processed for user `0ea6e3ba3f384e27aad3fda7b50cf743`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:20.409751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bcd1b056-95c4-4f7c-a666-2df76f763201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:20.474267 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d5d0a-f0b6-49c3-9c9b-b4c9c486e491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:20.475805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d5d0a-f0b6-49c3-9c9b-b4c9c486e491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:20.476274 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d5d0a-f0b6-49c3-9c9b-b4c9c486e491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:20.476761 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d5d0a-f0b6-49c3-9c9b-b4c9c486e491 admin admin] PATH_INFO: `/v3/users/0caf1a659f074d089aba31bddee44e15` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:20.478949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d5d0a-f0b6-49c3-9c9b-b4c9c486e491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0caf1a659f074d089aba31bddee44e15)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:20.481802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d5d0a-f0b6-49c3-9c9b-b4c9c486e491 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:20.494320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1349/2701] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:54:20.508073 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1b056-95c4-4f7c-a666-2df76f763201 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:20.509862 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bcd1b056-95c4-4f7c-a666-2df76f763201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:20.582548 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1b056-95c4-4f7c-a666-2df76f763201 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:20.586481 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20004646-ac4f-44b3-b61c-a6c80c42cf12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:20.587147 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20004646-ac4f-44b3-b61c-a6c80c42cf12 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:20.587428 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20004646-ac4f-44b3-b61c-a6c80c42cf12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:20.587733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20004646-ac4f-44b3-b61c-a6c80c42cf12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:20.598936 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20004646-ac4f-44b3-b61c-a6c80c42cf12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:20.600454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20004646-ac4f-44b3-b61c-a6c80c42cf12 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:20.611643 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1350/2702] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:54:20.701252 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-857d5d0a-f0b6-49c3-9c9b-b4c9c486e491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0caf1a659f074d089aba31bddee44e15'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:20.701252 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-857d5d0a-f0b6-49c3-9c9b-b4c9c486e491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0caf1a659f074d089aba31bddee44e15'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:20.701802 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-857d5d0a-f0b6-49c3-9c9b-b4c9c486e491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0caf1a659f074d089aba31bddee44e15'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:20.707326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:20.717650 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-857d5d0a-f0b6-49c3-9c9b-b4c9c486e491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0caf1a659f074d089aba31bddee44e15'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:20.727201 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-857d5d0a-f0b6-49c3-9c9b-b4c9c486e491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0caf1a659f074d089aba31bddee44e15'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:20.835795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1353/2703] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:54:20 2022] DELETE /identity/v3/users/0caf1a659f074d089aba31bddee44e15 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:54:21.319513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20004646-ac4f-44b3-b61c-a6c80c42cf12 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:21.333016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20004646-ac4f-44b3-b61c-a6c80c42cf12 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:21.375772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:21.393215 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc2534-c9ba-49ea-bcf2-1b65741a6996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:21.394081 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc2534-c9ba-49ea-bcf2-1b65741a6996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:21.394361 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc2534-c9ba-49ea-bcf2-1b65741a6996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:21.394633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc2534-c9ba-49ea-bcf2-1b65741a6996 admin admin] PATH_INFO: `/v3/projects/39ec7a9a14cd41aa97d4d4ddb16c21fc` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:21.395913 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc2534-c9ba-49ea-bcf2-1b65741a6996 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39ec7a9a14cd41aa97d4d4ddb16c21fc)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:21.397801 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc2534-c9ba-49ea-bcf2-1b65741a6996 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:21.527409 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-34bc2534-c9ba-49ea-bcf2-1b65741a6996 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39ec7a9a14cd41aa97d4d4ddb16c21fc', 'request_id': 'req-34bc2534-c9ba-49ea-bcf2-1b65741a6996'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:21.537271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-34bc2534-c9ba-49ea-bcf2-1b65741a6996 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39ec7a9a14cd41aa97d4d4ddb16c21fc', 'request_id': 'req-34bc2534-c9ba-49ea-bcf2-1b65741a6996'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:21.543246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-34bc2534-c9ba-49ea-bcf2-1b65741a6996 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39ec7a9a14cd41aa97d4d4ddb16c21fc', 'request_id': 'req-34bc2534-c9ba-49ea-bcf2-1b65741a6996'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:21.661944 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-34bc2534-c9ba-49ea-bcf2-1b65741a6996 admin admin] The token cache is being invalidate because project 39ec7a9a14cd41aa97d4d4ddb16c21fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:54:21.663005 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-34bc2534-c9ba-49ea-bcf2-1b65741a6996 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:21.663005 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-34bc2534-c9ba-49ea-bcf2-1b65741a6996 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:21.667747 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1351/2704] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:54:21 2022] DELETE /identity/v3/projects/39ec7a9a14cd41aa97d4d4ddb16c21fc => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:21.672823 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03d11e-5080-42ae-b233-3feabb43738b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:21.673410 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03d11e-5080-42ae-b233-3feabb43738b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:21.674061 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03d11e-5080-42ae-b233-3feabb43738b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:21.710661 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-8c03d11e-5080-42ae-b233-3feabb43738b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:21.712752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8c03d11e-5080-42ae-b233-3feabb43738b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:21.813511 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1354/2705] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:21.830816 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd241c-1b1c-43f0-a849-762da203ab1b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:21.831110 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd241c-1b1c-43f0-a849-762da203ab1b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:21.831501 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd241c-1b1c-43f0-a849-762da203ab1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:21.867746 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-4ccd241c-1b1c-43f0-a849-762da203ab1b None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:21.869772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4ccd241c-1b1c-43f0-a849-762da203ab1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:21.883785 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1352/2706] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:54:21.890135 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03d11e-5080-42ae-b233-3feabb43738b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:21.907903 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03d11e-5080-42ae-b233-3feabb43738b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:21.912148 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c662e-1e1f-43db-bd74-35d0e96eb138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:21.912148 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c662e-1e1f-43db-bd74-35d0e96eb138 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:21.912148 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c662e-1e1f-43db-bd74-35d0e96eb138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:21.912148 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c662e-1e1f-43db-bd74-35d0e96eb138 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:21.912839 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c662e-1e1f-43db-bd74-35d0e96eb138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:54:21.912839 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c662e-1e1f-43db-bd74-35d0e96eb138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:21.914523 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c662e-1e1f-43db-bd74-35d0e96eb138 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:21.918699 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd241c-1b1c-43f0-a849-762da203ab1b None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:21.920384 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4ccd241c-1b1c-43f0-a849-762da203ab1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:21.930888 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1355/2707] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:54:21.937209 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c662e-1e1f-43db-bd74-35d0e96eb138 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:21.959769 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c662e-1e1f-43db-bd74-35d0e96eb138 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:21.968424 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d3dd9-4e5d-4983-bdfe-a2e1b5e1737f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:21.969712 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d3dd9-4e5d-4983-bdfe-a2e1b5e1737f admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:21.970276 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d3dd9-4e5d-4983-bdfe-a2e1b5e1737f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:21.970862 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d3dd9-4e5d-4983-bdfe-a2e1b5e1737f admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:21.972391 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d3dd9-4e5d-4983-bdfe-a2e1b5e1737f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:21.975897 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d3dd9-4e5d-4983-bdfe-a2e1b5e1737f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:21.985923 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd241c-1b1c-43f0-a849-762da203ab1b None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:21.989860 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ec1cf-270c-4fb6-851e-845569c10e32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:21.990661 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ec1cf-270c-4fb6-851e-845569c10e32 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:21.990840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ec1cf-270c-4fb6-851e-845569c10e32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:21.991042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ec1cf-270c-4fb6-851e-845569c10e32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:21.993235 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8f4ec1cf-270c-4fb6-851e-845569c10e32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:22.062230 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ec1cf-270c-4fb6-851e-845569c10e32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:22.063712 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ec1cf-270c-4fb6-851e-845569c10e32 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:22.074287 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1353/2708] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:54:22.135937 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1356/2709] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:54:21 2022] POST /identity/v3/projects => generated 369 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:54:22.141019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ec1cf-270c-4fb6-851e-845569c10e32 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:22.153980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ec1cf-270c-4fb6-851e-845569c10e32 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:22.193192 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:22.221777 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf910c53-07b8-450f-8ad6-fe2d81408463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:22.222832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf910c53-07b8-450f-8ad6-fe2d81408463 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:22.223307 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf910c53-07b8-450f-8ad6-fe2d81408463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:22.223817 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf910c53-07b8-450f-8ad6-fe2d81408463 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:22.224752 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf910c53-07b8-450f-8ad6-fe2d81408463 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:22.226924 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf910c53-07b8-450f-8ad6-fe2d81408463 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:22.475484 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1354/2710] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:54:22 2022] POST /identity/v3/users => generated 285 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:54:22.480036 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d3dd9-4e5d-4983-bdfe-a2e1b5e1737f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:22.491456 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d3dd9-4e5d-4983-bdfe-a2e1b5e1737f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:22.496370 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1fb64-b6e6-4f47-8cb4-658660cb24fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:22.497074 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1fb64-b6e6-4f47-8cb4-658660cb24fd admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:22.497374 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1fb64-b6e6-4f47-8cb4-658660cb24fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:22.497672 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1fb64-b6e6-4f47-8cb4-658660cb24fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:22.498741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1fb64-b6e6-4f47-8cb4-658660cb24fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:22.500439 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1fb64-b6e6-4f47-8cb4-658660cb24fd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:22.511923 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1357/2711] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:54:22.516832 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf910c53-07b8-450f-8ad6-fe2d81408463 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:22.528576 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf910c53-07b8-450f-8ad6-fe2d81408463 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:22.534258 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae6cc3-aca5-4f5e-a227-b3f87e026fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:22.534952 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae6cc3-aca5-4f5e-a227-b3f87e026fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:22.535238 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae6cc3-aca5-4f5e-a227-b3f87e026fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:22.535516 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae6cc3-aca5-4f5e-a227-b3f87e026fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:22.536757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae6cc3-aca5-4f5e-a227-b3f87e026fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:22.537994 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae6cc3-aca5-4f5e-a227-b3f87e026fa3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:22.550262 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1355/2712] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:54:22.555419 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1fb64-b6e6-4f47-8cb4-658660cb24fd admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:22.566681 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1fb64-b6e6-4f47-8cb4-658660cb24fd admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:22.571495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f409b-1232-4aeb-bf18-8d2db1bae1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:22.572558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f409b-1232-4aeb-bf18-8d2db1bae1b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:22.572844 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f409b-1232-4aeb-bf18-8d2db1bae1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:22.573129 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f409b-1232-4aeb-bf18-8d2db1bae1b8 admin admin] PATH_INFO: `/v3/projects/af9ad438347d41cd908b67516b2c05f9/users/34549cd34b95499b8d60c9c60139d40f/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:22.589898 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f409b-1232-4aeb-bf18-8d2db1bae1b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af9ad438347d41cd908b67516b2c05f9, user_id=34549cd34b95499b8d60c9c60139d40f, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:22.593656 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f409b-1232-4aeb-bf18-8d2db1bae1b8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:22.688555 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1358/2713] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:54:22 2022] PUT /identity/v3/projects/af9ad438347d41cd908b67516b2c05f9/users/34549cd34b95499b8d60c9c60139d40f/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:22.699690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25547a42-67a1-4734-bc8c-89567f6e5213 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:22.700464 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25547a42-67a1-4734-bc8c-89567f6e5213 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:22.700726 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25547a42-67a1-4734-bc8c-89567f6e5213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:22.734305 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-25547a42-67a1-4734-bc8c-89567f6e5213 None None] MFA Rules not processed for user `34549cd34b95499b8d60c9c60139d40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:22.735913 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-25547a42-67a1-4734-bc8c-89567f6e5213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:22.820118 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1356/2714] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:22 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:22.835076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f409b-1232-4aeb-bf18-8d2db1bae1b8 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:22.836957 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-fc0f409b-1232-4aeb-bf18-8d2db1bae1b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:22.908347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f409b-1232-4aeb-bf18-8d2db1bae1b8 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:22.912282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26d647-6524-4ab6-910a-1b05cd2419d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '694088ed30b94bad9d3b16f3766252df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:22.912964 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26d647-6524-4ab6-910a-1b05cd2419d7 service glance] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:22.913245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26d647-6524-4ab6-910a-1b05cd2419d7 service glance] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:22.913517 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26d647-6524-4ab6-910a-1b05cd2419d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:22.923862 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd26d647-6524-4ab6-910a-1b05cd2419d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:22.925342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd26d647-6524-4ab6-910a-1b05cd2419d7 service glance] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:22.937134 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1359/2715] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:22 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:54:25.712270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41666e1-9901-4126-a802-3c119770f6d0 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:25.713196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41666e1-9901-4126-a802-3c119770f6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:25.713196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41666e1-9901-4126-a802-3c119770f6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:25.738066 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-b41666e1-9901-4126-a802-3c119770f6d0 None None] MFA Rules not processed for user `4e8cc08181c74bb584fdca44adba662a`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:25.740277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b41666e1-9901-4126-a802-3c119770f6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:25.820574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1357/2716] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:54:27.778104 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26d647-6524-4ab6-910a-1b05cd2419d7 service glance] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:27.780921 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-dd26d647-6524-4ab6-910a-1b05cd2419d7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:27.787882 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:27.864858 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26d647-6524-4ab6-910a-1b05cd2419d7 service glance] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:27.868790 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1d80c-6782-41b3-98e5-d5069db91f3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:27.869465 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1d80c-6782-41b3-98e5-d5069db91f3d service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:27.869750 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1d80c-6782-41b3-98e5-d5069db91f3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:27.870026 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1d80c-6782-41b3-98e5-d5069db91f3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:27.871980 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-44f1d80c-6782-41b3-98e5-d5069db91f3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:27.957250 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1d80c-6782-41b3-98e5-d5069db91f3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:27.960292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1d80c-6782-41b3-98e5-d5069db91f3d service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:27.977529 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1360/2717] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:54:33.266315 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41666e1-9901-4126-a802-3c119770f6d0 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:33.267888 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b41666e1-9901-4126-a802-3c119770f6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:33.313442 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:33.352937 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41666e1-9901-4126-a802-3c119770f6d0 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:33.359424 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22006a-9044-4abb-8d24-dc5d8a04b4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:33.360513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22006a-9044-4abb-8d24-dc5d8a04b4b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:33.360719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22006a-9044-4abb-8d24-dc5d8a04b4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:33.360996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22006a-9044-4abb-8d24-dc5d8a04b4b8 admin admin] PATH_INFO: `/v3/users/955812fb4b284187ba9e6d0d3e165d0c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:33.362310 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22006a-9044-4abb-8d24-dc5d8a04b4b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=955812fb4b284187ba9e6d0d3e165d0c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:33.364426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22006a-9044-4abb-8d24-dc5d8a04b4b8 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:33.594636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5a22006a-9044-4abb-8d24-dc5d8a04b4b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '955812fb4b284187ba9e6d0d3e165d0c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:33.595327 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5a22006a-9044-4abb-8d24-dc5d8a04b4b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '955812fb4b284187ba9e6d0d3e165d0c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:33.595969 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5a22006a-9044-4abb-8d24-dc5d8a04b4b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '955812fb4b284187ba9e6d0d3e165d0c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:33.612443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5a22006a-9044-4abb-8d24-dc5d8a04b4b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '955812fb4b284187ba9e6d0d3e165d0c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:33.623216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-5a22006a-9044-4abb-8d24-dc5d8a04b4b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '955812fb4b284187ba9e6d0d3e165d0c'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:33.711233 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1358/2718] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:54:33 2022] DELETE /identity/v3/users/955812fb4b284187ba9e6d0d3e165d0c => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:54:33.721483 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfea29-31d4-46d8-9831-e81bce5e8981 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:33.722152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfea29-31d4-46d8-9831-e81bce5e8981 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:33.722152 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfea29-31d4-46d8-9831-e81bce5e8981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:33.742917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-94cfea29-31d4-46d8-9831-e81bce5e8981 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:33.744231 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-94cfea29-31d4-46d8-9831-e81bce5e8981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:33.820060 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1361/2719] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:54:33.831571 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22006a-9044-4abb-8d24-dc5d8a04b4b8 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:33.834060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5a22006a-9044-4abb-8d24-dc5d8a04b4b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:33.970728 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22006a-9044-4abb-8d24-dc5d8a04b4b8 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:33.977805 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae539b33-198a-400c-bbc3-c0461f9b345f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:33.978982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae539b33-198a-400c-bbc3-c0461f9b345f service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:33.979472 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae539b33-198a-400c-bbc3-c0461f9b345f service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:33.980036 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae539b33-198a-400c-bbc3-c0461f9b345f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:33.994503 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae539b33-198a-400c-bbc3-c0461f9b345f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:33.998441 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae539b33-198a-400c-bbc3-c0461f9b345f service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:34.022445 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1359/2720] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:54:34.614193 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfea29-31d4-46d8-9831-e81bce5e8981 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:34.616010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-94cfea29-31d4-46d8-9831-e81bce5e8981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:34.631847 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfea29-31d4-46d8-9831-e81bce5e8981 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:34.639224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524295d1-14d0-4216-b969-4d96f5dcdc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:34.641872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524295d1-14d0-4216-b969-4d96f5dcdc8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:34.642363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524295d1-14d0-4216-b969-4d96f5dcdc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:34.642862 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524295d1-14d0-4216-b969-4d96f5dcdc8d admin admin] PATH_INFO: `/v3/projects/636d96e7383a4dbd9d8c5f30c1c31b0c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:34.645505 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524295d1-14d0-4216-b969-4d96f5dcdc8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=636d96e7383a4dbd9d8c5f30c1c31b0c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:34.649980 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524295d1-14d0-4216-b969-4d96f5dcdc8d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:34.830378 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-524295d1-14d0-4216-b969-4d96f5dcdc8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '636d96e7383a4dbd9d8c5f30c1c31b0c', 'request_id': 'req-524295d1-14d0-4216-b969-4d96f5dcdc8d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:34.833994 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:34.840353 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-524295d1-14d0-4216-b969-4d96f5dcdc8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '636d96e7383a4dbd9d8c5f30c1c31b0c', 'request_id': 'req-524295d1-14d0-4216-b969-4d96f5dcdc8d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:34.851209 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-524295d1-14d0-4216-b969-4d96f5dcdc8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '636d96e7383a4dbd9d8c5f30c1c31b0c', 'request_id': 'req-524295d1-14d0-4216-b969-4d96f5dcdc8d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:35.129291 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-524295d1-14d0-4216-b969-4d96f5dcdc8d admin admin] The token cache is being invalidate because project 636d96e7383a4dbd9d8c5f30c1c31b0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:54:35.129578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-524295d1-14d0-4216-b969-4d96f5dcdc8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:35.130334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-524295d1-14d0-4216-b969-4d96f5dcdc8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:35.132105 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1362/2721] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:54:34 2022] DELETE /identity/v3/projects/636d96e7383a4dbd9d8c5f30c1c31b0c => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:35.136270 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae539b33-198a-400c-bbc3-c0461f9b345f service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:35.148284 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae539b33-198a-400c-bbc3-c0461f9b345f service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:35.209840 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:35.217829 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9c8c1-efcd-45cb-8f0a-09f12fa28534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:35.218733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9c8c1-efcd-45cb-8f0a-09f12fa28534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:35.219020 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9c8c1-efcd-45cb-8f0a-09f12fa28534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:35.219306 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9c8c1-efcd-45cb-8f0a-09f12fa28534 admin admin] PATH_INFO: `/v3/users/0cf300eecafd42c4888671f9872010f4` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:35.220729 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e9c8c1-efcd-45cb-8f0a-09f12fa28534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cf300eecafd42c4888671f9872010f4)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:35.222825 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e9c8c1-efcd-45cb-8f0a-09f12fa28534 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:35.752496 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a1e9c8c1-efcd-45cb-8f0a-09f12fa28534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cf300eecafd42c4888671f9872010f4'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:35.753159 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a1e9c8c1-efcd-45cb-8f0a-09f12fa28534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cf300eecafd42c4888671f9872010f4'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:35.753758 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a1e9c8c1-efcd-45cb-8f0a-09f12fa28534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cf300eecafd42c4888671f9872010f4'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:35.767046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a1e9c8c1-efcd-45cb-8f0a-09f12fa28534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cf300eecafd42c4888671f9872010f4'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:35.775375 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-a1e9c8c1-efcd-45cb-8f0a-09f12fa28534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cf300eecafd42c4888671f9872010f4'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:35.974641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1360/2722] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:54:35 2022] DELETE /identity/v3/users/0cf300eecafd42c4888671f9872010f4 => generated 0 bytes in 839 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:36.511075 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524295d1-14d0-4216-b969-4d96f5dcdc8d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:36.515766 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-524295d1-14d0-4216-b969-4d96f5dcdc8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:36.541236 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524295d1-14d0-4216-b969-4d96f5dcdc8d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:36.666725 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61299ce-30a9-4844-af7c-946fafb327fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:36.668861 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61299ce-30a9-4844-af7c-946fafb327fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:36.670915 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61299ce-30a9-4844-af7c-946fafb327fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:36.671326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61299ce-30a9-4844-af7c-946fafb327fe admin admin] PATH_INFO: `/v3/projects/0f392356c22b44798ceb3725d62656bb` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:36.673801 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61299ce-30a9-4844-af7c-946fafb327fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f392356c22b44798ceb3725d62656bb)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:36.679303 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61299ce-30a9-4844-af7c-946fafb327fe admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:36.846904 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b61299ce-30a9-4844-af7c-946fafb327fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f392356c22b44798ceb3725d62656bb', 'request_id': 'req-b61299ce-30a9-4844-af7c-946fafb327fe'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:36.865984 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b61299ce-30a9-4844-af7c-946fafb327fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f392356c22b44798ceb3725d62656bb', 'request_id': 'req-b61299ce-30a9-4844-af7c-946fafb327fe'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:36.879050 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b61299ce-30a9-4844-af7c-946fafb327fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f392356c22b44798ceb3725d62656bb', 'request_id': 'req-b61299ce-30a9-4844-af7c-946fafb327fe'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:36.949204 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b61299ce-30a9-4844-af7c-946fafb327fe admin admin] The token cache is being invalidate because project 0f392356c22b44798ceb3725d62656bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:54:36.949633 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b61299ce-30a9-4844-af7c-946fafb327fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:36.951582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-b61299ce-30a9-4844-af7c-946fafb327fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:36.953711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1363/2723] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:54:36 2022] DELETE /identity/v3/projects/0f392356c22b44798ceb3725d62656bb => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:36.958628 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9c8c1-efcd-45cb-8f0a-09f12fa28534 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:36.960572 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a1e9c8c1-efcd-45cb-8f0a-09f12fa28534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:36.976042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9c8c1-efcd-45cb-8f0a-09f12fa28534 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:37.053149 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9128bb-764d-48cb-b661-3b38800c08f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:37.054063 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9128bb-764d-48cb-b661-3b38800c08f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:37.054357 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9128bb-764d-48cb-b661-3b38800c08f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:37.054627 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9128bb-764d-48cb-b661-3b38800c08f5 admin admin] PATH_INFO: `/v3/users/0ea6e3ba3f384e27aad3fda7b50cf743` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:37.056018 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9128bb-764d-48cb-b661-3b38800c08f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ea6e3ba3f384e27aad3fda7b50cf743)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:37.058173 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9128bb-764d-48cb-b661-3b38800c08f5 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:37.285638 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2b9128bb-764d-48cb-b661-3b38800c08f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ea6e3ba3f384e27aad3fda7b50cf743'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:37.286667 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2b9128bb-764d-48cb-b661-3b38800c08f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ea6e3ba3f384e27aad3fda7b50cf743'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:37.287544 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2b9128bb-764d-48cb-b661-3b38800c08f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ea6e3ba3f384e27aad3fda7b50cf743'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:37.305830 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2b9128bb-764d-48cb-b661-3b38800c08f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ea6e3ba3f384e27aad3fda7b50cf743'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:37.313992 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-2b9128bb-764d-48cb-b661-3b38800c08f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ea6e3ba3f384e27aad3fda7b50cf743'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:37.316720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:37.435539 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1361/2724] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:54:36 2022] DELETE /identity/v3/users/0ea6e3ba3f384e27aad3fda7b50cf743 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:38.051878 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61299ce-30a9-4844-af7c-946fafb327fe admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:38.054453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b61299ce-30a9-4844-af7c-946fafb327fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:38.074393 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61299ce-30a9-4844-af7c-946fafb327fe admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:38.180359 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9adb5b0-c797-454d-9f83-f13ad769421d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:38.181912 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adb5b0-c797-454d-9f83-f13ad769421d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:38.182446 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adb5b0-c797-454d-9f83-f13ad769421d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:38.182977 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adb5b0-c797-454d-9f83-f13ad769421d admin admin] PATH_INFO: `/v3/projects/5d620400aff64ba28fd099f288900f64` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:38.185297 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9adb5b0-c797-454d-9f83-f13ad769421d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d620400aff64ba28fd099f288900f64)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:38.187840 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9adb5b0-c797-454d-9f83-f13ad769421d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:38.204075 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:38.328430 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e9adb5b0-c797-454d-9f83-f13ad769421d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d620400aff64ba28fd099f288900f64', 'request_id': 'req-e9adb5b0-c797-454d-9f83-f13ad769421d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:38.334573 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e9adb5b0-c797-454d-9f83-f13ad769421d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d620400aff64ba28fd099f288900f64', 'request_id': 'req-e9adb5b0-c797-454d-9f83-f13ad769421d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:38.343548 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e9adb5b0-c797-454d-9f83-f13ad769421d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d620400aff64ba28fd099f288900f64', 'request_id': 'req-e9adb5b0-c797-454d-9f83-f13ad769421d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:38.431493 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e9adb5b0-c797-454d-9f83-f13ad769421d admin admin] The token cache is being invalidate because project 5d620400aff64ba28fd099f288900f64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:54:38.431862 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e9adb5b0-c797-454d-9f83-f13ad769421d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:38.432666 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-e9adb5b0-c797-454d-9f83-f13ad769421d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:38.434444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1364/2725] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:54:38 2022] DELETE /identity/v3/projects/5d620400aff64ba28fd099f288900f64 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:38.441822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168a4f0-4b6e-4a1d-badc-33aea52e5a78 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:38.442442 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168a4f0-4b6e-4a1d-badc-33aea52e5a78 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:38.442775 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168a4f0-4b6e-4a1d-badc-33aea52e5a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:38.468196 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-8168a4f0-4b6e-4a1d-badc-33aea52e5a78 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:38.470238 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8168a4f0-4b6e-4a1d-badc-33aea52e5a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:38.577086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1362/2726] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:38.592144 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2ac28-38e4-4d97-b011-814b965d6327 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:38.592665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2ac28-38e4-4d97-b011-814b965d6327 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:38.593188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2ac28-38e4-4d97-b011-814b965d6327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:38.623978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7db2ac28-38e4-4d97-b011-814b965d6327 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:38.626108 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7db2ac28-38e4-4d97-b011-814b965d6327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:38.638655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1365/2727] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:38.643364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168a4f0-4b6e-4a1d-badc-33aea52e5a78 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:38.655143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168a4f0-4b6e-4a1d-badc-33aea52e5a78 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:38.660641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf647b-5266-4bc1-b9cf-5a59c3dea52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:38.661470 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf647b-5266-4bc1-b9cf-5a59c3dea52c admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:38.661899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf647b-5266-4bc1-b9cf-5a59c3dea52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:38.662342 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf647b-5266-4bc1-b9cf-5a59c3dea52c admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:38.663216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf647b-5266-4bc1-b9cf-5a59c3dea52c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:54:38.663634 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf647b-5266-4bc1-b9cf-5a59c3dea52c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:38.665555 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf647b-5266-4bc1-b9cf-5a59c3dea52c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:38.685111 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1363/2728] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:54:38.687178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2ac28-38e4-4d97-b011-814b965d6327 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:38.698220 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2ac28-38e4-4d97-b011-814b965d6327 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:38.704177 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abd433-54ab-45a1-83b8-e1025ff31657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:38.704871 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abd433-54ab-45a1-83b8-e1025ff31657 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:38.705161 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abd433-54ab-45a1-83b8-e1025ff31657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:38.705441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abd433-54ab-45a1-83b8-e1025ff31657 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:38.706156 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46abd433-54ab-45a1-83b8-e1025ff31657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:38.708141 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46abd433-54ab-45a1-83b8-e1025ff31657 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:38.811612 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1366/2729] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:54:38 2022] POST /identity/v3/projects => generated 393 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:54:38.816378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf647b-5266-4bc1-b9cf-5a59c3dea52c admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:38.828594 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf647b-5266-4bc1-b9cf-5a59c3dea52c admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:38.908667 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9c903-f143-4003-b983-2c0619e171a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:38.909847 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9c903-f143-4003-b983-2c0619e171a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:38.910278 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9c903-f143-4003-b983-2c0619e171a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:38.910601 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9c903-f143-4003-b983-2c0619e171a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:38.911428 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9c903-f143-4003-b983-2c0619e171a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:38.913740 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9c903-f143-4003-b983-2c0619e171a3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:39.075280 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1364/2730] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:54:38 2022] POST /identity/v3/users => generated 297 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:54:39.079718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abd433-54ab-45a1-83b8-e1025ff31657 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:39.100268 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abd433-54ab-45a1-83b8-e1025ff31657 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:39.109598 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fb7d2-92ae-452a-a10c-06d0ef847f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:39.110695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fb7d2-92ae-452a-a10c-06d0ef847f5c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:39.111201 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fb7d2-92ae-452a-a10c-06d0ef847f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:39.111650 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fb7d2-92ae-452a-a10c-06d0ef847f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:39.113116 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fb7d2-92ae-452a-a10c-06d0ef847f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:39.116138 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fb7d2-92ae-452a-a10c-06d0ef847f5c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:39.137222 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1367/2731] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:54:39.143630 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9c903-f143-4003-b983-2c0619e171a3 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:39.163145 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9c903-f143-4003-b983-2c0619e171a3 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:39.171854 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afaec6d-d8d0-4c84-948e-e50315b60e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:39.173255 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaec6d-d8d0-4c84-948e-e50315b60e6f admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:39.173683 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaec6d-d8d0-4c84-948e-e50315b60e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:39.174045 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaec6d-d8d0-4c84-948e-e50315b60e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:39.175172 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afaec6d-d8d0-4c84-948e-e50315b60e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:39.176871 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afaec6d-d8d0-4c84-948e-e50315b60e6f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:39.194852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1365/2732] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:54:39.196703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fb7d2-92ae-452a-a10c-06d0ef847f5c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:39.216461 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fb7d2-92ae-452a-a10c-06d0ef847f5c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:39.226780 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858dd1f-2ea5-47a5-bdb2-f0373056dd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:39.228898 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858dd1f-2ea5-47a5-bdb2-f0373056dd32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:39.228898 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858dd1f-2ea5-47a5-bdb2-f0373056dd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:39.229284 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858dd1f-2ea5-47a5-bdb2-f0373056dd32 admin admin] PATH_INFO: `/v3/projects/d1d1ba423c2844be9997d879e797a882/users/118fcd26101443668d24bd595baf90de/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:39.259724 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858dd1f-2ea5-47a5-bdb2-f0373056dd32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d1ba423c2844be9997d879e797a882, user_id=118fcd26101443668d24bd595baf90de, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:39.266431 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858dd1f-2ea5-47a5-bdb2-f0373056dd32 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:39.416747 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1368/2733] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:54:39 2022] PUT /identity/v3/projects/d1d1ba423c2844be9997d879e797a882/users/118fcd26101443668d24bd595baf90de/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:54:39.431219 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dfc16-7dda-4761-a79f-8a41c9c592f1 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:39.435533 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dfc16-7dda-4761-a79f-8a41c9c592f1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:39.435533 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dfc16-7dda-4761-a79f-8a41c9c592f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:39.465312 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0e0dfc16-7dda-4761-a79f-8a41c9c592f1 None None] MFA Rules not processed for user `118fcd26101443668d24bd595baf90de`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:39.466899 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0e0dfc16-7dda-4761-a79f-8a41c9c592f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:39.530780 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:39.553206 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1366/2734] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:39.558127 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858dd1f-2ea5-47a5-bdb2-f0373056dd32 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:39.641733 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858dd1f-2ea5-47a5-bdb2-f0373056dd32 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:39.647484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61d028-548f-4777-8ed5-8746b114d9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:39.648417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61d028-548f-4777-8ed5-8746b114d9f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:39.648875 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61d028-548f-4777-8ed5-8746b114d9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:39.649202 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61d028-548f-4777-8ed5-8746b114d9f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:39.649995 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61d028-548f-4777-8ed5-8746b114d9f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:39.652388 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61d028-548f-4777-8ed5-8746b114d9f4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:39.980580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1369/2735] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:54:39 2022] POST /identity/v3/projects => generated 391 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:54:39.983057 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0dfc16-7dda-4761-a79f-8a41c9c592f1 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:39.994457 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0dfc16-7dda-4761-a79f-8a41c9c592f1 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:40.060722 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e07a9-4f9e-4665-a7ab-09005cf6ce71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:40.061399 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e07a9-4f9e-4665-a7ab-09005cf6ce71 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:40.061682 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e07a9-4f9e-4665-a7ab-09005cf6ce71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:40.061954 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e07a9-4f9e-4665-a7ab-09005cf6ce71 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:40.062643 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e07a9-4f9e-4665-a7ab-09005cf6ce71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:40.065010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e07a9-4f9e-4665-a7ab-09005cf6ce71 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:40.230646 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1367/2736] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:54:39 2022] POST /identity/v3/users => generated 302 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:54:40.236558 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61d028-548f-4777-8ed5-8746b114d9f4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:40.247645 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61d028-548f-4777-8ed5-8746b114d9f4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:40.253569 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ab762-9032-46be-967a-05cbe83e3e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:40.254407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ab762-9032-46be-967a-05cbe83e3e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:40.254707 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ab762-9032-46be-967a-05cbe83e3e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:40.254968 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ab762-9032-46be-967a-05cbe83e3e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:40.255819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ab762-9032-46be-967a-05cbe83e3e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:40.257498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ab762-9032-46be-967a-05cbe83e3e12 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:40.277845 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1370/2737] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:54:40.283023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e07a9-4f9e-4665-a7ab-09005cf6ce71 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:40.294619 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e07a9-4f9e-4665-a7ab-09005cf6ce71 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:40.300301 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c58ac-a364-4fd4-9be6-808d379590ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:40.301275 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c58ac-a364-4fd4-9be6-808d379590ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:40.301628 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c58ac-a364-4fd4-9be6-808d379590ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:40.301914 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c58ac-a364-4fd4-9be6-808d379590ec admin admin] PATH_INFO: `/v3/projects/e976c1b8b91543659e0a3212f87d76dd/users/219ac76273d2406ba229390953739820/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:40.318642 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c58ac-a364-4fd4-9be6-808d379590ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e976c1b8b91543659e0a3212f87d76dd, user_id=219ac76273d2406ba229390953739820, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:40.321489 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c58ac-a364-4fd4-9be6-808d379590ec admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:40.467700 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1368/2738] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:54:40 2022] PUT /identity/v3/projects/e976c1b8b91543659e0a3212f87d76dd/users/219ac76273d2406ba229390953739820/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:40.472275 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ab762-9032-46be-967a-05cbe83e3e12 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:40.484225 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ab762-9032-46be-967a-05cbe83e3e12 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:40.492816 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:40.550417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17b841-8e63-4d12-b185-c55603645e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:40.551279 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17b841-8e63-4d12-b185-c55603645e58 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:40.551569 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17b841-8e63-4d12-b185-c55603645e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:40.551880 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17b841-8e63-4d12-b185-c55603645e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:40.552635 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17b841-8e63-4d12-b185-c55603645e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:40.554230 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17b841-8e63-4d12-b185-c55603645e58 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:40.565828 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1371/2739] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:54:40.570174 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c58ac-a364-4fd4-9be6-808d379590ec admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:40.581469 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c58ac-a364-4fd4-9be6-808d379590ec admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:40.586129 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ee8b9-a775-4c5d-ab6c-1a0891d67235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:40.587005 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ee8b9-a775-4c5d-ab6c-1a0891d67235 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:40.587289 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ee8b9-a775-4c5d-ab6c-1a0891d67235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:40.587564 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ee8b9-a775-4c5d-ab6c-1a0891d67235 admin admin] PATH_INFO: `/v3/projects/e976c1b8b91543659e0a3212f87d76dd/users/219ac76273d2406ba229390953739820/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:40.589562 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ee8b9-a775-4c5d-ab6c-1a0891d67235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e976c1b8b91543659e0a3212f87d76dd, user_id=219ac76273d2406ba229390953739820, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:40.593303 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ee8b9-a775-4c5d-ab6c-1a0891d67235 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:40.604935 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1369/2740] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:54:40 2022] PUT /identity/v3/projects/e976c1b8b91543659e0a3212f87d76dd/users/219ac76273d2406ba229390953739820/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:40.616505 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cc9c2-8a61-43a8-b6b4-d2ece75bec2b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:40.616806 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cc9c2-8a61-43a8-b6b4-d2ece75bec2b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:40.617092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cc9c2-8a61-43a8-b6b4-d2ece75bec2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:40.647726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-813cc9c2-8a61-43a8-b6b4-d2ece75bec2b None None] MFA Rules not processed for user `219ac76273d2406ba229390953739820`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:40.648989 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-813cc9c2-8a61-43a8-b6b4-d2ece75bec2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:40.774530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1372/2741] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:40.791150 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ee8b9-a775-4c5d-ab6c-1a0891d67235 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:40.793571 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-631ee8b9-a775-4c5d-ab6c-1a0891d67235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:40.870322 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ee8b9-a775-4c5d-ab6c-1a0891d67235 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:40.874957 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edabae-ede1-4c28-968a-78e897eb519b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:40.875794 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edabae-ede1-4c28-968a-78e897eb519b service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:40.876195 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edabae-ede1-4c28-968a-78e897eb519b service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:40.876521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edabae-ede1-4c28-968a-78e897eb519b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:40.888001 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edabae-ede1-4c28-968a-78e897eb519b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:40.890118 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edabae-ede1-4c28-968a-78e897eb519b service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:40.902964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1370/2742] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:54:40.962164 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813cc9c2-8a61-43a8-b6b4-d2ece75bec2b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:40.983498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813cc9c2-8a61-43a8-b6b4-d2ece75bec2b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:40.988877 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40a865-7d15-4d9a-9d9a-53756e9c0029 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:40.989839 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40a865-7d15-4d9a-9d9a-53756e9c0029 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:40.990186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40a865-7d15-4d9a-9d9a-53756e9c0029 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:40.990503 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40a865-7d15-4d9a-9d9a-53756e9c0029 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:41.065920 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40a865-7d15-4d9a-9d9a-53756e9c0029 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:41.067525 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40a865-7d15-4d9a-9d9a-53756e9c0029 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:41.079542 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1373/2743] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:54:48.653706 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edabae-ede1-4c28-968a-78e897eb519b service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:48.734120 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:48.754940 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edabae-ede1-4c28-968a-78e897eb519b service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:48.760102 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2305c-6a48-4de4-b9d5-418019737452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:48.760994 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2305c-6a48-4de4-b9d5-418019737452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:48.761325 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2305c-6a48-4de4-b9d5-418019737452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:48.761569 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2305c-6a48-4de4-b9d5-418019737452 admin admin] PATH_INFO: `/v3/users/118fcd26101443668d24bd595baf90de` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:48.762933 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2305c-6a48-4de4-b9d5-418019737452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=118fcd26101443668d24bd595baf90de)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:48.766004 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2305c-6a48-4de4-b9d5-418019737452 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:49.081234 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6fa2305c-6a48-4de4-b9d5-418019737452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '118fcd26101443668d24bd595baf90de'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:49.081926 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6fa2305c-6a48-4de4-b9d5-418019737452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '118fcd26101443668d24bd595baf90de'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:49.082546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6fa2305c-6a48-4de4-b9d5-418019737452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '118fcd26101443668d24bd595baf90de'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:49.098372 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6fa2305c-6a48-4de4-b9d5-418019737452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '118fcd26101443668d24bd595baf90de'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:49.107115 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6fa2305c-6a48-4de4-b9d5-418019737452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '118fcd26101443668d24bd595baf90de'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:49.244200 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1371/2744] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:54:48 2022] DELETE /identity/v3/users/118fcd26101443668d24bd595baf90de => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:49.251300 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434631f-b477-460a-b41f-a44d6e0a0c45 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:49.252052 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434631f-b477-460a-b41f-a44d6e0a0c45 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:49.252653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434631f-b477-460a-b41f-a44d6e0a0c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:49.283341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a434631f-b477-460a-b41f-a44d6e0a0c45 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:49.285105 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a434631f-b477-460a-b41f-a44d6e0a0c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:49.326605 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:49.385446 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1374/2745] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:54:49.396258 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2305c-6a48-4de4-b9d5-418019737452 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:49.397970 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6fa2305c-6a48-4de4-b9d5-418019737452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:49.476810 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2305c-6a48-4de4-b9d5-418019737452 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:49.481366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190a1e8-5945-4c09-925b-4783477362a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:49.482262 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190a1e8-5945-4c09-925b-4783477362a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:49.482670 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190a1e8-5945-4c09-925b-4783477362a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:49.483597 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190a1e8-5945-4c09-925b-4783477362a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:49.494820 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190a1e8-5945-4c09-925b-4783477362a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:49.496471 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190a1e8-5945-4c09-925b-4783477362a6 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:49.508677 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1372/2746] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:54:50.051778 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434631f-b477-460a-b41f-a44d6e0a0c45 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:50.053682 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a434631f-b477-460a-b41f-a44d6e0a0c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:50.069959 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434631f-b477-460a-b41f-a44d6e0a0c45 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:50.077260 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af232ee4-d23d-4d6a-94d2-5c5c8cd8efa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:50.078253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af232ee4-d23d-4d6a-94d2-5c5c8cd8efa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:50.078408 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af232ee4-d23d-4d6a-94d2-5c5c8cd8efa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:50.078692 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af232ee4-d23d-4d6a-94d2-5c5c8cd8efa0 admin admin] PATH_INFO: `/v3/projects/d1d1ba423c2844be9997d879e797a882` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:50.080082 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af232ee4-d23d-4d6a-94d2-5c5c8cd8efa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1d1ba423c2844be9997d879e797a882)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:50.082190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af232ee4-d23d-4d6a-94d2-5c5c8cd8efa0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:50.216834 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-af232ee4-d23d-4d6a-94d2-5c5c8cd8efa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1d1ba423c2844be9997d879e797a882', 'request_id': 'req-af232ee4-d23d-4d6a-94d2-5c5c8cd8efa0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:50.229565 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-af232ee4-d23d-4d6a-94d2-5c5c8cd8efa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1d1ba423c2844be9997d879e797a882', 'request_id': 'req-af232ee4-d23d-4d6a-94d2-5c5c8cd8efa0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:50.243343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-af232ee4-d23d-4d6a-94d2-5c5c8cd8efa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1d1ba423c2844be9997d879e797a882', 'request_id': 'req-af232ee4-d23d-4d6a-94d2-5c5c8cd8efa0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:50.340901 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-af232ee4-d23d-4d6a-94d2-5c5c8cd8efa0 admin admin] The token cache is being invalidate because project d1d1ba423c2844be9997d879e797a882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:54:50.342095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-af232ee4-d23d-4d6a-94d2-5c5c8cd8efa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:50.342095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-af232ee4-d23d-4d6a-94d2-5c5c8cd8efa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:50.344184 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1375/2747] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:54:50 2022] DELETE /identity/v3/projects/d1d1ba423c2844be9997d879e797a882 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:50.349371 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190a1e8-5945-4c09-925b-4783477362a6 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:50.366885 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190a1e8-5945-4c09-925b-4783477362a6 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:50.446346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7915442-b3cb-46d5-8cfb-f73161a63be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:50.447237 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7915442-b3cb-46d5-8cfb-f73161a63be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:50.447517 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7915442-b3cb-46d5-8cfb-f73161a63be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:50.447828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7915442-b3cb-46d5-8cfb-f73161a63be3 admin admin] PATH_INFO: `/v3/users/219ac76273d2406ba229390953739820` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:50.449097 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7915442-b3cb-46d5-8cfb-f73161a63be3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=219ac76273d2406ba229390953739820)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:50.451319 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7915442-b3cb-46d5-8cfb-f73161a63be3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:50.458982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:50.636002 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b7915442-b3cb-46d5-8cfb-f73161a63be3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '219ac76273d2406ba229390953739820'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:50.636856 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b7915442-b3cb-46d5-8cfb-f73161a63be3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '219ac76273d2406ba229390953739820'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:50.637539 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b7915442-b3cb-46d5-8cfb-f73161a63be3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '219ac76273d2406ba229390953739820'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:50.653255 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b7915442-b3cb-46d5-8cfb-f73161a63be3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '219ac76273d2406ba229390953739820'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:50.662504 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b7915442-b3cb-46d5-8cfb-f73161a63be3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '219ac76273d2406ba229390953739820'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:50.737915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1373/2748] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:54:50 2022] DELETE /identity/v3/users/219ac76273d2406ba229390953739820 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:51.191235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af232ee4-d23d-4d6a-94d2-5c5c8cd8efa0 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:51.196956 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-af232ee4-d23d-4d6a-94d2-5c5c8cd8efa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:51.212085 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af232ee4-d23d-4d6a-94d2-5c5c8cd8efa0 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:51.286190 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3e2c1-2fbd-4916-9690-74ccdf08609f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:51.287041 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3e2c1-2fbd-4916-9690-74ccdf08609f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:51.287329 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3e2c1-2fbd-4916-9690-74ccdf08609f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:51.287610 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3e2c1-2fbd-4916-9690-74ccdf08609f admin admin] PATH_INFO: `/v3/projects/e976c1b8b91543659e0a3212f87d76dd` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:51.289001 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3e2c1-2fbd-4916-9690-74ccdf08609f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e976c1b8b91543659e0a3212f87d76dd)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:51.290975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3e2c1-2fbd-4916-9690-74ccdf08609f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:51.440722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-58d3e2c1-2fbd-4916-9690-74ccdf08609f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e976c1b8b91543659e0a3212f87d76dd', 'request_id': 'req-58d3e2c1-2fbd-4916-9690-74ccdf08609f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:51.448406 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:51.450596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-58d3e2c1-2fbd-4916-9690-74ccdf08609f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e976c1b8b91543659e0a3212f87d76dd', 'request_id': 'req-58d3e2c1-2fbd-4916-9690-74ccdf08609f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:51.463870 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-58d3e2c1-2fbd-4916-9690-74ccdf08609f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e976c1b8b91543659e0a3212f87d76dd', 'request_id': 'req-58d3e2c1-2fbd-4916-9690-74ccdf08609f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:51.542315 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-58d3e2c1-2fbd-4916-9690-74ccdf08609f admin admin] The token cache is being invalidate because project e976c1b8b91543659e0a3212f87d76dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:54:51.542597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-58d3e2c1-2fbd-4916-9690-74ccdf08609f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:51.543179 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-58d3e2c1-2fbd-4916-9690-74ccdf08609f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:51.544809 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1376/2749] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:54:51 2022] DELETE /identity/v3/projects/e976c1b8b91543659e0a3212f87d76dd => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:51.550298 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccb00e-24ff-47a2-87b0-1ff99bbe3774 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:51.550822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccb00e-24ff-47a2-87b0-1ff99bbe3774 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:51.550822 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccb00e-24ff-47a2-87b0-1ff99bbe3774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:51.570755 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-23ccb00e-24ff-47a2-87b0-1ff99bbe3774 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:51.572026 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-23ccb00e-24ff-47a2-87b0-1ff99bbe3774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:51.641772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1374/2750] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:51.651533 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466320c-97cc-4283-877b-97d355130760 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:51.651859 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466320c-97cc-4283-877b-97d355130760 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:51.652139 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466320c-97cc-4283-877b-97d355130760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:51.672507 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-b466320c-97cc-4283-877b-97d355130760 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:51.673756 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-b466320c-97cc-4283-877b-97d355130760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:51.682064 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1377/2751] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:54:51.687284 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccb00e-24ff-47a2-87b0-1ff99bbe3774 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:51.697439 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccb00e-24ff-47a2-87b0-1ff99bbe3774 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:51.703339 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98019eca-7537-46d9-8a3e-5d40b044084d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:51.704040 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98019eca-7537-46d9-8a3e-5d40b044084d admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:51.704322 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98019eca-7537-46d9-8a3e-5d40b044084d admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:51.704595 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98019eca-7537-46d9-8a3e-5d40b044084d admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:51.705330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98019eca-7537-46d9-8a3e-5d40b044084d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:54:51.705632 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98019eca-7537-46d9-8a3e-5d40b044084d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:51.707222 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98019eca-7537-46d9-8a3e-5d40b044084d admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:51.721537 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1375/2752] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:54:51.726740 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466320c-97cc-4283-877b-97d355130760 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:51.743130 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466320c-97cc-4283-877b-97d355130760 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:51.752186 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f7218-7f43-4270-adcf-e9867144c93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:51.753321 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f7218-7f43-4270-adcf-e9867144c93d admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:51.753787 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f7218-7f43-4270-adcf-e9867144c93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:51.754287 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f7218-7f43-4270-adcf-e9867144c93d admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:51.755622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f7218-7f43-4270-adcf-e9867144c93d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:51.758394 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f7218-7f43-4270-adcf-e9867144c93d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:51.856188 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1378/2753] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:54:51 2022] POST /identity/v3/projects => generated 361 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:54:51.861229 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98019eca-7537-46d9-8a3e-5d40b044084d admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:51.877919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98019eca-7537-46d9-8a3e-5d40b044084d admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:51.947459 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fed1a5-68ae-453b-8b4f-52cfb72f75ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:51.948231 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed1a5-68ae-453b-8b4f-52cfb72f75ca admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:51.948514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed1a5-68ae-453b-8b4f-52cfb72f75ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:51.948785 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed1a5-68ae-453b-8b4f-52cfb72f75ca admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:51.949482 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fed1a5-68ae-453b-8b4f-52cfb72f75ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:51.952406 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fed1a5-68ae-453b-8b4f-52cfb72f75ca admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:52.183428 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1376/2754] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:54:51 2022] POST /identity/v3/users => generated 281 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:54:52.187635 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f7218-7f43-4270-adcf-e9867144c93d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:52.198412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f7218-7f43-4270-adcf-e9867144c93d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:52.204348 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a3490-9826-41c1-ad36-af0c7e10d208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:52.222341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a3490-9826-41c1-ad36-af0c7e10d208 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:52.222341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a3490-9826-41c1-ad36-af0c7e10d208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:52.222341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a3490-9826-41c1-ad36-af0c7e10d208 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:52.222341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a3490-9826-41c1-ad36-af0c7e10d208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:52.222341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a3490-9826-41c1-ad36-af0c7e10d208 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:52.222341 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1379/2755] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:54:52.225744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fed1a5-68ae-453b-8b4f-52cfb72f75ca admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:52.231259 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:52.238923 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fed1a5-68ae-453b-8b4f-52cfb72f75ca admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:52.243961 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39942f3-c2ee-4725-aa6a-8c67640a6f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:52.244658 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39942f3-c2ee-4725-aa6a-8c67640a6f9a admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:52.244944 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39942f3-c2ee-4725-aa6a-8c67640a6f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:52.245224 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39942f3-c2ee-4725-aa6a-8c67640a6f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:52.245960 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39942f3-c2ee-4725-aa6a-8c67640a6f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:52.247569 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39942f3-c2ee-4725-aa6a-8c67640a6f9a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:52.259304 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1377/2756] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:54:52.264564 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a3490-9826-41c1-ad36-af0c7e10d208 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:52.276255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a3490-9826-41c1-ad36-af0c7e10d208 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:52.282049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad30bae-3807-4975-bdf4-4b1ae9bd0006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:52.282977 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad30bae-3807-4975-bdf4-4b1ae9bd0006 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:52.283263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad30bae-3807-4975-bdf4-4b1ae9bd0006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:52.283544 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad30bae-3807-4975-bdf4-4b1ae9bd0006 admin admin] PATH_INFO: `/v3/projects/068e6fcd6278468fad0ead6a85f639f5/users/584e270caafc425580f598ff561bd053/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:52.300157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad30bae-3807-4975-bdf4-4b1ae9bd0006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=068e6fcd6278468fad0ead6a85f639f5, user_id=584e270caafc425580f598ff561bd053, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:52.302872 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad30bae-3807-4975-bdf4-4b1ae9bd0006 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:52.598225 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1380/2757] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:54:52 2022] PUT /identity/v3/projects/068e6fcd6278468fad0ead6a85f639f5/users/584e270caafc425580f598ff561bd053/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:52.609785 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437cb8e-4c8d-44b5-a0b1-9e66c25d8090 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:52.610317 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437cb8e-4c8d-44b5-a0b1-9e66c25d8090 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:52.610721 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437cb8e-4c8d-44b5-a0b1-9e66c25d8090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:52.639454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-f437cb8e-4c8d-44b5-a0b1-9e66c25d8090 None None] MFA Rules not processed for user `584e270caafc425580f598ff561bd053`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:52.640848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f437cb8e-4c8d-44b5-a0b1-9e66c25d8090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:52.720505 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1378/2758] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:52.755788 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad30bae-3807-4975-bdf4-4b1ae9bd0006 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:52.758246 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2ad30bae-3807-4975-bdf4-4b1ae9bd0006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:52.834497 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad30bae-3807-4975-bdf4-4b1ae9bd0006 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:52.839261 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354521e-c981-4f79-a631-96261f4a51f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:52.840010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354521e-c981-4f79-a631-96261f4a51f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:52.840386 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354521e-c981-4f79-a631-96261f4a51f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:52.840677 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354521e-c981-4f79-a631-96261f4a51f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:52.852168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354521e-c981-4f79-a631-96261f4a51f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:52.853652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354521e-c981-4f79-a631-96261f4a51f7 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:52.865085 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1381/2759] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:54:52.881056 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437cb8e-4c8d-44b5-a0b1-9e66c25d8090 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:52.950865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437cb8e-4c8d-44b5-a0b1-9e66c25d8090 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:52.956750 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031585c-383f-459f-9886-f236d4e0f4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:52.957447 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031585c-383f-459f-9886-f236d4e0f4dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:52.957733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031585c-383f-459f-9886-f236d4e0f4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:52.957987 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031585c-383f-459f-9886-f236d4e0f4dc admin admin] PATH_INFO: `/v3/users/584e270caafc425580f598ff561bd053` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:52.959571 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b031585c-383f-459f-9886-f236d4e0f4dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=584e270caafc425580f598ff561bd053)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:52.962468 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b031585c-383f-459f-9886-f236d4e0f4dc admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:53.278728 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b031585c-383f-459f-9886-f236d4e0f4dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '584e270caafc425580f598ff561bd053'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:53.279394 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b031585c-383f-459f-9886-f236d4e0f4dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '584e270caafc425580f598ff561bd053'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:53.280664 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b031585c-383f-459f-9886-f236d4e0f4dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '584e270caafc425580f598ff561bd053'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:53.293695 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b031585c-383f-459f-9886-f236d4e0f4dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '584e270caafc425580f598ff561bd053'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:53.301460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b031585c-383f-459f-9886-f236d4e0f4dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '584e270caafc425580f598ff561bd053'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:53.415650 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:53.426175 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1379/2760] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:54:52 2022] DELETE /identity/v3/users/584e270caafc425580f598ff561bd053 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:53.429288 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe97e1-92e3-4c9d-be87-91829df5e6c9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:53.429586 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe97e1-92e3-4c9d-be87-91829df5e6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:53.429860 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe97e1-92e3-4c9d-be87-91829df5e6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:53.448529 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0dfe97e1-92e3-4c9d-be87-91829df5e6c9 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:53.449919 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0dfe97e1-92e3-4c9d-be87-91829df5e6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:53.520427 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1382/2761] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:53.531749 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031585c-383f-459f-9886-f236d4e0f4dc admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:53.533783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b031585c-383f-459f-9886-f236d4e0f4dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:53.603021 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031585c-383f-459f-9886-f236d4e0f4dc admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:53.606815 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e93a98-5fb4-4e8a-8c1d-ba1be4f334d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:53.607483 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e93a98-5fb4-4e8a-8c1d-ba1be4f334d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:53.607824 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e93a98-5fb4-4e8a-8c1d-ba1be4f334d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:53.608240 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e93a98-5fb4-4e8a-8c1d-ba1be4f334d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:53.618432 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e93a98-5fb4-4e8a-8c1d-ba1be4f334d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:53.619902 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e93a98-5fb4-4e8a-8c1d-ba1be4f334d2 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:53.630396 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1380/2762] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:54:54.083318 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe97e1-92e3-4c9d-be87-91829df5e6c9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:54.085652 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0dfe97e1-92e3-4c9d-be87-91829df5e6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:54.104680 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe97e1-92e3-4c9d-be87-91829df5e6c9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:54.110867 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef74034-6497-4ddc-8663-67ac9205a3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:54.112485 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef74034-6497-4ddc-8663-67ac9205a3da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:54.112485 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef74034-6497-4ddc-8663-67ac9205a3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:54.112485 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef74034-6497-4ddc-8663-67ac9205a3da admin admin] PATH_INFO: `/v3/projects/068e6fcd6278468fad0ead6a85f639f5` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:54.113924 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef74034-6497-4ddc-8663-67ac9205a3da admin admin] RBAC: Authorizing `identity:delete_project(project_id=068e6fcd6278468fad0ead6a85f639f5)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:54.116014 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef74034-6497-4ddc-8663-67ac9205a3da admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:54.143631 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:54.270362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cef74034-6497-4ddc-8663-67ac9205a3da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '068e6fcd6278468fad0ead6a85f639f5', 'request_id': 'req-cef74034-6497-4ddc-8663-67ac9205a3da'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:54.277206 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cef74034-6497-4ddc-8663-67ac9205a3da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '068e6fcd6278468fad0ead6a85f639f5', 'request_id': 'req-cef74034-6497-4ddc-8663-67ac9205a3da'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:54.285252 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cef74034-6497-4ddc-8663-67ac9205a3da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '068e6fcd6278468fad0ead6a85f639f5', 'request_id': 'req-cef74034-6497-4ddc-8663-67ac9205a3da'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:54.360337 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cef74034-6497-4ddc-8663-67ac9205a3da admin admin] The token cache is being invalidate because project 068e6fcd6278468fad0ead6a85f639f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:54:54.360609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cef74034-6497-4ddc-8663-67ac9205a3da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:54.361213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-cef74034-6497-4ddc-8663-67ac9205a3da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:54:54.362828 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1383/2763] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:54:54 2022] DELETE /identity/v3/projects/068e6fcd6278468fad0ead6a85f639f5 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:54.370502 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b491238-fdc4-4d14-b2a8-23c35be488e1 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:54.370964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b491238-fdc4-4d14-b2a8-23c35be488e1 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:54.371426 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b491238-fdc4-4d14-b2a8-23c35be488e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:54.392498 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-8b491238-fdc4-4d14-b2a8-23c35be488e1 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:54.393725 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8b491238-fdc4-4d14-b2a8-23c35be488e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:54.461367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1381/2764] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:54.470035 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5092b5-6f77-4118-9eef-56487953ef98 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:54.470376 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5092b5-6f77-4118-9eef-56487953ef98 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:54.470719 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5092b5-6f77-4118-9eef-56487953ef98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:54.493203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7d5092b5-6f77-4118-9eef-56487953ef98 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:54.494301 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7d5092b5-6f77-4118-9eef-56487953ef98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:54.503567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1384/2765] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:54.508156 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b491238-fdc4-4d14-b2a8-23c35be488e1 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:54.518967 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b491238-fdc4-4d14-b2a8-23c35be488e1 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:54.524333 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bffe4d-7edb-416e-82ef-9f74747f64ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:54.525020 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bffe4d-7edb-416e-82ef-9f74747f64ab admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:54.525308 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bffe4d-7edb-416e-82ef-9f74747f64ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:54.525588 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bffe4d-7edb-416e-82ef-9f74747f64ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:54.526343 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bffe4d-7edb-416e-82ef-9f74747f64ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:54:54.526655 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bffe4d-7edb-416e-82ef-9f74747f64ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:54.528354 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bffe4d-7edb-416e-82ef-9f74747f64ab admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:54.545007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1382/2766] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:54:54.555214 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5092b5-6f77-4118-9eef-56487953ef98 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:54.567352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5092b5-6f77-4118-9eef-56487953ef98 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:54.573882 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4af50-6877-497b-8292-821e79485c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:54.574868 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4af50-6877-497b-8292-821e79485c75 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:54.575284 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4af50-6877-497b-8292-821e79485c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:54.575460 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4af50-6877-497b-8292-821e79485c75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:54.576266 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca4af50-6877-497b-8292-821e79485c75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:54.578415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca4af50-6877-497b-8292-821e79485c75 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:54.665143 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1385/2767] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:54:54 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:54:54.669597 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bffe4d-7edb-416e-82ef-9f74747f64ab admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:54.680823 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bffe4d-7edb-416e-82ef-9f74747f64ab admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:54.744906 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aebace-c993-4ea7-8aa6-969446a0292c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:54.745584 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aebace-c993-4ea7-8aa6-969446a0292c admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:54.745868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aebace-c993-4ea7-8aa6-969446a0292c admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:54.746142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aebace-c993-4ea7-8aa6-969446a0292c admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:54.746842 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aebace-c993-4ea7-8aa6-969446a0292c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:54.748896 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aebace-c993-4ea7-8aa6-969446a0292c admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:54.834532 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1383/2768] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:54:54 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:54:54.838815 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4af50-6877-497b-8292-821e79485c75 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:54.850453 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4af50-6877-497b-8292-821e79485c75 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:54.855405 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c47a6-e23c-40c7-b58f-43440ef5a4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:54.856120 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c47a6-e23c-40c7-b58f-43440ef5a4ac admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:54.856405 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c47a6-e23c-40c7-b58f-43440ef5a4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:54.856683 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c47a6-e23c-40c7-b58f-43440ef5a4ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:54.857429 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c47a6-e23c-40c7-b58f-43440ef5a4ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:54.860086 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c47a6-e23c-40c7-b58f-43440ef5a4ac admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:54.871580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1386/2769] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:54:54.876627 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aebace-c993-4ea7-8aa6-969446a0292c admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:54.887366 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aebace-c993-4ea7-8aa6-969446a0292c admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:54.893018 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84996ca5-2c36-4460-aef3-0cd11ec893fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:54.893699 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84996ca5-2c36-4460-aef3-0cd11ec893fc admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:54.893984 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84996ca5-2c36-4460-aef3-0cd11ec893fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:54.894380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84996ca5-2c36-4460-aef3-0cd11ec893fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:54.894996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84996ca5-2c36-4460-aef3-0cd11ec893fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:54.896631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84996ca5-2c36-4460-aef3-0cd11ec893fc admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:54.907707 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1384/2770] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:54:54.912701 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c47a6-e23c-40c7-b58f-43440ef5a4ac admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:54.924002 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c47a6-e23c-40c7-b58f-43440ef5a4ac admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:54.928592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9d15f-4bf4-4c10-bda8-130f81d86369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:54.929474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9d15f-4bf4-4c10-bda8-130f81d86369 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:54.929756 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9d15f-4bf4-4c10-bda8-130f81d86369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:54.930034 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9d15f-4bf4-4c10-bda8-130f81d86369 admin admin] PATH_INFO: `/v3/projects/840cf582e6aa4391957d79242135b12f/users/cba91a03b02846298e1e1cb62b1765ea/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:54.945592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9d15f-4bf4-4c10-bda8-130f81d86369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=840cf582e6aa4391957d79242135b12f, user_id=cba91a03b02846298e1e1cb62b1765ea, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:54.948337 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9d15f-4bf4-4c10-bda8-130f81d86369 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:55.022901 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1387/2771] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:54:54 2022] PUT /identity/v3/projects/840cf582e6aa4391957d79242135b12f/users/cba91a03b02846298e1e1cb62b1765ea/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:55.042216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d320c-2794-43d8-b562-67c2a2c2a6aa None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:55.042712 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d320c-2794-43d8-b562-67c2a2c2a6aa None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:55.043187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d320c-2794-43d8-b562-67c2a2c2a6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:55.068852 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:55.081497 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-946d320c-2794-43d8-b562-67c2a2c2a6aa None None] MFA Rules not processed for user `cba91a03b02846298e1e1cb62b1765ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:55.082775 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-946d320c-2794-43d8-b562-67c2a2c2a6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:55.168944 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1385/2772] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:55.175976 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9d15f-4bf4-4c10-bda8-130f81d86369 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:55.259455 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9d15f-4bf4-4c10-bda8-130f81d86369 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:55.265203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bd008-a7a9-473e-a054-0a079936f73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:55.265893 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bd008-a7a9-473e-a054-0a079936f73e admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:55.266178 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bd008-a7a9-473e-a054-0a079936f73e admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:55.266456 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bd008-a7a9-473e-a054-0a079936f73e admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:55.267195 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0bd008-a7a9-473e-a054-0a079936f73e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:55.269245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0bd008-a7a9-473e-a054-0a079936f73e admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:55.378951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1388/2773] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:54:55 2022] POST /identity/v3/projects => generated 385 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:54:55.383529 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d320c-2794-43d8-b562-67c2a2c2a6aa None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:55.395274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d320c-2794-43d8-b562-67c2a2c2a6aa None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:55.458809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f5a35-6d2a-4aeb-bead-0b7ccb15fc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:55.459481 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f5a35-6d2a-4aeb-bead-0b7ccb15fc13 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:55.459796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f5a35-6d2a-4aeb-bead-0b7ccb15fc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:55.460075 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f5a35-6d2a-4aeb-bead-0b7ccb15fc13 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:55.460775 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f5a35-6d2a-4aeb-bead-0b7ccb15fc13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:55.462693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f5a35-6d2a-4aeb-bead-0b7ccb15fc13 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:55.549086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1386/2774] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:54:55 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:54:55.553998 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bd008-a7a9-473e-a054-0a079936f73e admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:55.565439 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bd008-a7a9-473e-a054-0a079936f73e admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:55.571409 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c5a51-79d1-4fae-a529-81541b89a9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:55.572199 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c5a51-79d1-4fae-a529-81541b89a9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:55.572474 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c5a51-79d1-4fae-a529-81541b89a9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:55.572764 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c5a51-79d1-4fae-a529-81541b89a9b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:55.573510 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c5a51-79d1-4fae-a529-81541b89a9b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:55.575396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c5a51-79d1-4fae-a529-81541b89a9b9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:55.587722 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1389/2775] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:54:55.591940 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f5a35-6d2a-4aeb-bead-0b7ccb15fc13 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:55.602719 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f5a35-6d2a-4aeb-bead-0b7ccb15fc13 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:55.607530 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a325f37-60c2-4d8d-add0-5dac06bafd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:55.608876 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a325f37-60c2-4d8d-add0-5dac06bafd63 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:55.609193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a325f37-60c2-4d8d-add0-5dac06bafd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:55.609470 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a325f37-60c2-4d8d-add0-5dac06bafd63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:55.610247 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a325f37-60c2-4d8d-add0-5dac06bafd63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:55.611889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a325f37-60c2-4d8d-add0-5dac06bafd63 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:55.622295 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1387/2776] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:54:55.626597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c5a51-79d1-4fae-a529-81541b89a9b9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:55.636577 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c5a51-79d1-4fae-a529-81541b89a9b9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:55.642027 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20de01d-f16e-4f75-8125-c41d8bbd3c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:55.642924 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20de01d-f16e-4f75-8125-c41d8bbd3c5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:55.643204 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20de01d-f16e-4f75-8125-c41d8bbd3c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:55.643475 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20de01d-f16e-4f75-8125-c41d8bbd3c5a admin admin] PATH_INFO: `/v3/projects/be79d6dc778640809a4a32b72fa877cc/users/c7bb37d33a9c4fd489142143a2ee874b/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:55.664561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20de01d-f16e-4f75-8125-c41d8bbd3c5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=be79d6dc778640809a4a32b72fa877cc, user_id=c7bb37d33a9c4fd489142143a2ee874b, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:55.667246 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20de01d-f16e-4f75-8125-c41d8bbd3c5a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:55.669698 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:55.754827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1390/2777] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:54:55 2022] PUT /identity/v3/projects/be79d6dc778640809a4a32b72fa877cc/users/c7bb37d33a9c4fd489142143a2ee874b/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:55.765584 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59c6e1-4f6c-4034-99af-7ef7492f4d48 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:55.766073 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59c6e1-4f6c-4034-99af-7ef7492f4d48 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:55.766193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59c6e1-4f6c-4034-99af-7ef7492f4d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:55.793087 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-ec59c6e1-4f6c-4034-99af-7ef7492f4d48 None None] MFA Rules not processed for user `c7bb37d33a9c4fd489142143a2ee874b`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:55.794304 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-ec59c6e1-4f6c-4034-99af-7ef7492f4d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:55.871927 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1388/2778] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:54:55.876519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20de01d-f16e-4f75-8125-c41d8bbd3c5a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:55.937760 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20de01d-f16e-4f75-8125-c41d8bbd3c5a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:55.942417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339d9f9-c9c8-461f-8c09-e9d497636dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:55.943121 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339d9f9-c9c8-461f-8c09-e9d497636dba admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:55.943718 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339d9f9-c9c8-461f-8c09-e9d497636dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:55.944197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339d9f9-c9c8-461f-8c09-e9d497636dba admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:55.945282 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339d9f9-c9c8-461f-8c09-e9d497636dba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:55.947309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339d9f9-c9c8-461f-8c09-e9d497636dba admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:56.052590 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1391/2779] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:54:55 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:54:56.056593 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59c6e1-4f6c-4034-99af-7ef7492f4d48 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:56.067478 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59c6e1-4f6c-4034-99af-7ef7492f4d48 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:56.071042 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:56.136774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c3a63-32d9-45ee-b790-8130d4ecdfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:56.137671 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c3a63-32d9-45ee-b790-8130d4ecdfe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:56.138099 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c3a63-32d9-45ee-b790-8130d4ecdfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:56.138430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c3a63-32d9-45ee-b790-8130d4ecdfe3 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:56.140624 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c3a63-32d9-45ee-b790-8130d4ecdfe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:56.143227 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c3a63-32d9-45ee-b790-8130d4ecdfe3 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:56.261216 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cee6b-0547-4080-881a-26b2e52286eb None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:56.261708 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cee6b-0547-4080-881a-26b2e52286eb None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:56.261827 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cee6b-0547-4080-881a-26b2e52286eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:56.282730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1389/2780] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:54:56 2022] POST /identity/v3/users => generated 299 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:54:56.283757 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d44cee6b-0547-4080-881a-26b2e52286eb None None] MFA Rules not processed for user `2b03f8b8c51e472bbd4cf0afc9c6b632`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:56.284073 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c3a63-32d9-45ee-b790-8130d4ecdfe3 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:56.285143 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d44cee6b-0547-4080-881a-26b2e52286eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:56.295506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c3a63-32d9-45ee-b790-8130d4ecdfe3 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:56.301402 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f49d7a-bd7a-4b03-bff4-a08a49fdce02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:56.302430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f49d7a-bd7a-4b03-bff4-a08a49fdce02 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:56.302430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f49d7a-bd7a-4b03-bff4-a08a49fdce02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:56.303056 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f49d7a-bd7a-4b03-bff4-a08a49fdce02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:56.303522 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f49d7a-bd7a-4b03-bff4-a08a49fdce02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:56.305247 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f49d7a-bd7a-4b03-bff4-a08a49fdce02 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:56.319921 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1390/2781] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:54:56.321158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f49d7a-bd7a-4b03-bff4-a08a49fdce02 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:56.332524 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f49d7a-bd7a-4b03-bff4-a08a49fdce02 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:56.339152 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7987f89-abaf-4978-b462-4fbf89f94992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:56.340373 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7987f89-abaf-4978-b462-4fbf89f94992 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:56.340815 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7987f89-abaf-4978-b462-4fbf89f94992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:56.341250 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7987f89-abaf-4978-b462-4fbf89f94992 admin admin] PATH_INFO: `/v3/projects/acc58895b5de45bfa9b683b76b3a3f90/users/09438fcf496146caba8b681fb1c2ae4b/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:56.350643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1392/2782] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Jul 18 08:54:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:54:56.359411 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7987f89-abaf-4978-b462-4fbf89f94992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc58895b5de45bfa9b683b76b3a3f90, user_id=09438fcf496146caba8b681fb1c2ae4b, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:56.362168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7987f89-abaf-4978-b462-4fbf89f94992 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:56.525034 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1391/2783] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:54:56 2022] PUT /identity/v3/projects/acc58895b5de45bfa9b683b76b3a3f90/users/09438fcf496146caba8b681fb1c2ae4b/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:54:56.529328 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cee6b-0547-4080-881a-26b2e52286eb None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:56.544582 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cee6b-0547-4080-881a-26b2e52286eb None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:56.550142 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7987f89-abaf-4978-b462-4fbf89f94992 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:56.551943 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d7987f89-abaf-4978-b462-4fbf89f94992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:56.615086 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cfc89-ab41-4df4-87c6-8774f8c6b1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:56.615811 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cfc89-ab41-4df4-87c6-8774f8c6b1ae admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:56.616171 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cfc89-ab41-4df4-87c6-8774f8c6b1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:56.618016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cfc89-ab41-4df4-87c6-8774f8c6b1ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:56.618016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cfc89-ab41-4df4-87c6-8774f8c6b1ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:56.618773 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cfc89-ab41-4df4-87c6-8774f8c6b1ae admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:56.632744 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1393/2784] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:54:56.636810 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cfc89-ab41-4df4-87c6-8774f8c6b1ae admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:56.648414 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cfc89-ab41-4df4-87c6-8774f8c6b1ae admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:56.651132 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7987f89-abaf-4978-b462-4fbf89f94992 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:56.653484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5c165-a3b9-43a3-b381-8f8e86c978a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:56.658412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5c165-a3b9-43a3-b381-8f8e86c978a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:56.658412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5c165-a3b9-43a3-b381-8f8e86c978a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:56.658412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5c165-a3b9-43a3-b381-8f8e86c978a0 admin admin] PATH_INFO: `/v3/projects/acc58895b5de45bfa9b683b76b3a3f90/users/09438fcf496146caba8b681fb1c2ae4b/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:56.658412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5c165-a3b9-43a3-b381-8f8e86c978a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc58895b5de45bfa9b683b76b3a3f90, user_id=09438fcf496146caba8b681fb1c2ae4b, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:56.658870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ff16c-6a54-47fe-9e8b-edde408b57e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09a5c2e804504038817cad76544dce5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:56.658870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ff16c-6a54-47fe-9e8b-edde408b57e9 service swift] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:56.658870 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ff16c-6a54-47fe-9e8b-edde408b57e9 service swift] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:56.659099 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ff16c-6a54-47fe-9e8b-edde408b57e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:56.660366 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5c165-a3b9-43a3-b381-8f8e86c978a0 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:56.676819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1394/2785] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:54:56 2022] PUT /identity/v3/projects/acc58895b5de45bfa9b683b76b3a3f90/users/09438fcf496146caba8b681fb1c2ae4b/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:54:56.685447 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec0b27-b9d9-4eb8-a2e3-aeb3e10ce34b None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:56.685744 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec0b27-b9d9-4eb8-a2e3-aeb3e10ce34b None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:56.686312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec0b27-b9d9-4eb8-a2e3-aeb3e10ce34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:56.700715 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:56.718539 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-30ec0b27-b9d9-4eb8-a2e3-aeb3e10ce34b None None] MFA Rules not processed for user `09438fcf496146caba8b681fb1c2ae4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:54:56.721674 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-30ec0b27-b9d9-4eb8-a2e3-aeb3e10ce34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:56.725768 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ff16c-6a54-47fe-9e8b-edde408b57e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:56.727365 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ff16c-6a54-47fe-9e8b-edde408b57e9 service swift] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:56.736051 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1392/2786] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Jul 18 08:54:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 08:54:56.820971 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1395/2787] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:54:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:54:56.839736 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ff16c-6a54-47fe-9e8b-edde408b57e9 service swift] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:56.839736 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-518ff16c-6a54-47fe-9e8b-edde408b57e9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:54:56.852566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:54:56.908367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ff16c-6a54-47fe-9e8b-edde408b57e9 service swift] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:56.967843 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af3efb-9eae-4b51-a14e-0033f5e58504 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:56.968774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af3efb-9eae-4b51-a14e-0033f5e58504 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:56.969076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af3efb-9eae-4b51-a14e-0033f5e58504 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:56.969331 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af3efb-9eae-4b51-a14e-0033f5e58504 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:57.050511 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af3efb-9eae-4b51-a14e-0033f5e58504 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:57.052336 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af3efb-9eae-4b51-a14e-0033f5e58504 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:57.065271 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1393/2788] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:54:59.095868 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec0b27-b9d9-4eb8-a2e3-aeb3e10ce34b None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:54:59.109224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec0b27-b9d9-4eb8-a2e3-aeb3e10ce34b None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:54:59.116155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659725d4-da99-4115-86ea-b3dc8dfe853a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:54:59.117352 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659725d4-da99-4115-86ea-b3dc8dfe853a service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:54:59.117893 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659725d4-da99-4115-86ea-b3dc8dfe853a service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:54:59.118421 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659725d4-da99-4115-86ea-b3dc8dfe853a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:54:59.237059 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659725d4-da99-4115-86ea-b3dc8dfe853a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:54:59.239782 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659725d4-da99-4115-86ea-b3dc8dfe853a service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:54:59.253669 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1396/2789] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:54:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:55:01.266246 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af3efb-9eae-4b51-a14e-0033f5e58504 service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:01.275220 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af3efb-9eae-4b51-a14e-0033f5e58504 service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:01.279573 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3ab89-0dc0-4ec5-9b07-a4a3ba25e6a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:01.281078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3ab89-0dc0-4ec5-9b07-a4a3ba25e6a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:01.281405 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3ab89-0dc0-4ec5-9b07-a4a3ba25e6a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:01.281785 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3ab89-0dc0-4ec5-9b07-a4a3ba25e6a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:01.292669 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da3ab89-0dc0-4ec5-9b07-a4a3ba25e6a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:01.294381 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da3ab89-0dc0-4ec5-9b07-a4a3ba25e6a8 service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:01.306917 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1394/2790] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:55:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:55:01.596529 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659725d4-da99-4115-86ea-b3dc8dfe853a service cinder] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:01.598876 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-659725d4-da99-4115-86ea-b3dc8dfe853a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:01.694258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659725d4-da99-4115-86ea-b3dc8dfe853a service cinder] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:01.700342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a057a-78d6-4ef5-be0d-50076f7b1639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:01.702031 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a057a-78d6-4ef5-be0d-50076f7b1639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:01.702598 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a057a-78d6-4ef5-be0d-50076f7b1639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:01.703109 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a057a-78d6-4ef5-be0d-50076f7b1639 admin admin] PATH_INFO: `/v3/users/34549cd34b95499b8d60c9c60139d40f` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:01.704686 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a057a-78d6-4ef5-be0d-50076f7b1639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34549cd34b95499b8d60c9c60139d40f)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:01.708973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a057a-78d6-4ef5-be0d-50076f7b1639 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:01.935405 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:01.960928 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ea3a057a-78d6-4ef5-be0d-50076f7b1639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34549cd34b95499b8d60c9c60139d40f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:01.962807 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ea3a057a-78d6-4ef5-be0d-50076f7b1639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34549cd34b95499b8d60c9c60139d40f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:01.964027 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ea3a057a-78d6-4ef5-be0d-50076f7b1639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34549cd34b95499b8d60c9c60139d40f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:01.988017 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ea3a057a-78d6-4ef5-be0d-50076f7b1639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34549cd34b95499b8d60c9c60139d40f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:01.999512 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-ea3a057a-78d6-4ef5-be0d-50076f7b1639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34549cd34b95499b8d60c9c60139d40f'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:02.082391 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1397/2791] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:55:01 2022] DELETE /identity/v3/users/34549cd34b95499b8d60c9c60139d40f => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:55:02.087398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c768ef6-b2ac-4a2d-9a16-de94ebbec106 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:02.087977 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c768ef6-b2ac-4a2d-9a16-de94ebbec106 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:02.088364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c768ef6-b2ac-4a2d-9a16-de94ebbec106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:02.112878 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-6c768ef6-b2ac-4a2d-9a16-de94ebbec106 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:02.114211 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6c768ef6-b2ac-4a2d-9a16-de94ebbec106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:02.192664 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:02.205889 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1395/2792] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:02.219167 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a057a-78d6-4ef5-be0d-50076f7b1639 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:02.221430 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ea3a057a-78d6-4ef5-be0d-50076f7b1639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:02.312732 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a057a-78d6-4ef5-be0d-50076f7b1639 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:02.316487 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956c6b9-7d7d-4caa-abcd-eee0e33da3c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:02.317207 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956c6b9-7d7d-4caa-abcd-eee0e33da3c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:02.319084 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956c6b9-7d7d-4caa-abcd-eee0e33da3c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:02.319084 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956c6b9-7d7d-4caa-abcd-eee0e33da3c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:02.331966 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956c6b9-7d7d-4caa-abcd-eee0e33da3c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:02.334295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956c6b9-7d7d-4caa-abcd-eee0e33da3c9 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:02.346475 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1398/2793] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:55:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:55:03.014935 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c768ef6-b2ac-4a2d-9a16-de94ebbec106 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:03.014935 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6c768ef6-b2ac-4a2d-9a16-de94ebbec106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:03.014935 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c768ef6-b2ac-4a2d-9a16-de94ebbec106 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:03.016478 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe91753-9fdf-4bce-84d1-92aa419deff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:03.016478 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe91753-9fdf-4bce-84d1-92aa419deff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:03.016478 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe91753-9fdf-4bce-84d1-92aa419deff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:03.016478 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe91753-9fdf-4bce-84d1-92aa419deff4 admin admin] PATH_INFO: `/v3/projects/af9ad438347d41cd908b67516b2c05f9` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:03.017269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe91753-9fdf-4bce-84d1-92aa419deff4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af9ad438347d41cd908b67516b2c05f9)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:03.017269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe91753-9fdf-4bce-84d1-92aa419deff4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:03.201040 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7fe91753-9fdf-4bce-84d1-92aa419deff4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af9ad438347d41cd908b67516b2c05f9', 'request_id': 'req-7fe91753-9fdf-4bce-84d1-92aa419deff4'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:03.207393 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7fe91753-9fdf-4bce-84d1-92aa419deff4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af9ad438347d41cd908b67516b2c05f9', 'request_id': 'req-7fe91753-9fdf-4bce-84d1-92aa419deff4'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:03.216162 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7fe91753-9fdf-4bce-84d1-92aa419deff4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af9ad438347d41cd908b67516b2c05f9', 'request_id': 'req-7fe91753-9fdf-4bce-84d1-92aa419deff4'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:03.312144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7fe91753-9fdf-4bce-84d1-92aa419deff4 admin admin] The token cache is being invalidate because project af9ad438347d41cd908b67516b2c05f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:55:03.312144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7fe91753-9fdf-4bce-84d1-92aa419deff4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:03.312144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-7fe91753-9fdf-4bce-84d1-92aa419deff4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:03.312144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1396/2794] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:55:02 2022] DELETE /identity/v3/projects/af9ad438347d41cd908b67516b2c05f9 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:55:03.316295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd81622-d7bd-4058-87a6-2bef5c827413 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:03.317793 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd81622-d7bd-4058-87a6-2bef5c827413 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:03.318496 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd81622-d7bd-4058-87a6-2bef5c827413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:03.346570 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-2bd81622-d7bd-4058-87a6-2bef5c827413 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:03.347826 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2bd81622-d7bd-4058-87a6-2bef5c827413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:03.437809 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1399/2795] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:03.446446 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3d1fc-ef92-4729-bd9f-e8d078f419ca None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:03.447783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3d1fc-ef92-4729-bd9f-e8d078f419ca None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:03.447783 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3d1fc-ef92-4729-bd9f-e8d078f419ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:03.474610 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2fb3d1fc-ef92-4729-bd9f-e8d078f419ca None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:03.476598 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2fb3d1fc-ef92-4729-bd9f-e8d078f419ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:03.487328 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1397/2796] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:03.493199 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd81622-d7bd-4058-87a6-2bef5c827413 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:03.504736 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd81622-d7bd-4058-87a6-2bef5c827413 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:03.510115 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a72b4-8bd1-4471-8058-c765794dd8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:03.510952 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a72b4-8bd1-4471-8058-c765794dd8b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:03.511245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a72b4-8bd1-4471-8058-c765794dd8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:03.511524 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a72b4-8bd1-4471-8058-c765794dd8b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:03.512322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a72b4-8bd1-4471-8058-c765794dd8b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:55:03.512637 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a72b4-8bd1-4471-8058-c765794dd8b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:03.514253 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a72b4-8bd1-4471-8058-c765794dd8b3 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:03.533425 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1400/2797] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:55:03.539632 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3d1fc-ef92-4729-bd9f-e8d078f419ca None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:03.552841 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3d1fc-ef92-4729-bd9f-e8d078f419ca None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:03.558095 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a44798-8589-46c9-8f0a-d3c1e3ce6776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:03.558773 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a44798-8589-46c9-8f0a-d3c1e3ce6776 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:03.559060 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a44798-8589-46c9-8f0a-d3c1e3ce6776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:03.559338 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a44798-8589-46c9-8f0a-d3c1e3ce6776 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:03.560080 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a44798-8589-46c9-8f0a-d3c1e3ce6776 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:03.562907 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a44798-8589-46c9-8f0a-d3c1e3ce6776 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:03.685254 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1398/2798] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:55:03 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:55:03.690268 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a72b4-8bd1-4471-8058-c765794dd8b3 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:03.703623 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a72b4-8bd1-4471-8058-c765794dd8b3 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:03.751383 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:03.786734 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf96e7-8df9-491b-9cf4-31cbb985fdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:03.791452 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf96e7-8df9-491b-9cf4-31cbb985fdfd admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:03.791452 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf96e7-8df9-491b-9cf4-31cbb985fdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:03.791452 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf96e7-8df9-491b-9cf4-31cbb985fdfd admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:03.791452 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf96e7-8df9-491b-9cf4-31cbb985fdfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:03.792775 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf96e7-8df9-491b-9cf4-31cbb985fdfd admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:03.936055 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1401/2799] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:55:03 2022] POST /identity/v3/users => generated 292 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:55:03.940170 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a44798-8589-46c9-8f0a-d3c1e3ce6776 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:03.954795 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a44798-8589-46c9-8f0a-d3c1e3ce6776 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:03.962076 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94391da4-325b-49f1-a8ce-90f99f9fcfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:03.962856 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94391da4-325b-49f1-a8ce-90f99f9fcfa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:03.963148 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94391da4-325b-49f1-a8ce-90f99f9fcfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:03.963454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94391da4-325b-49f1-a8ce-90f99f9fcfa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:03.964413 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94391da4-325b-49f1-a8ce-90f99f9fcfa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:03.966183 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94391da4-325b-49f1-a8ce-90f99f9fcfa4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:03.979722 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1399/2800] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:55:03.987881 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf96e7-8df9-491b-9cf4-31cbb985fdfd admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:04.009098 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf96e7-8df9-491b-9cf4-31cbb985fdfd admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:04.019484 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ecbea-0796-4d6c-8f0e-174d6da8fe29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:04.020791 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ecbea-0796-4d6c-8f0e-174d6da8fe29 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:04.021396 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ecbea-0796-4d6c-8f0e-174d6da8fe29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:04.021819 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ecbea-0796-4d6c-8f0e-174d6da8fe29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:04.023945 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ecbea-0796-4d6c-8f0e-174d6da8fe29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:04.027407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ecbea-0796-4d6c-8f0e-174d6da8fe29 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:04.047477 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1402/2801] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:55:04.054119 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94391da4-325b-49f1-a8ce-90f99f9fcfa4 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:04.077814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94391da4-325b-49f1-a8ce-90f99f9fcfa4 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:04.098560 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0348bf9f-c08c-4392-94e9-9cf81b3b9c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:04.103178 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0348bf9f-c08c-4392-94e9-9cf81b3b9c67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:04.105043 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0348bf9f-c08c-4392-94e9-9cf81b3b9c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:04.107192 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0348bf9f-c08c-4392-94e9-9cf81b3b9c67 admin admin] PATH_INFO: `/v3/projects/c75ce285523c420a89891d9520d9595e/users/f7ce7458121343d0a77cc49579a3d744/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:04.151517 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0348bf9f-c08c-4392-94e9-9cf81b3b9c67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c75ce285523c420a89891d9520d9595e, user_id=f7ce7458121343d0a77cc49579a3d744, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:04.157814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0348bf9f-c08c-4392-94e9-9cf81b3b9c67 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:04.306249 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1400/2802] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:55:04 2022] PUT /identity/v3/projects/c75ce285523c420a89891d9520d9595e/users/f7ce7458121343d0a77cc49579a3d744/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:55:04.381741 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20752377-0c1d-45b4-bdf1-01f20721b411 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:04.382723 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20752377-0c1d-45b4-bdf1-01f20721b411 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:04.384100 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20752377-0c1d-45b4-bdf1-01f20721b411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:04.421262 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-20752377-0c1d-45b4-bdf1-01f20721b411 None None] MFA Rules not processed for user `f7ce7458121343d0a77cc49579a3d744`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:04.423034 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-20752377-0c1d-45b4-bdf1-01f20721b411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:04.528168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1403/2803] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:04.539207 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0348bf9f-c08c-4392-94e9-9cf81b3b9c67 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:04.542559 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0348bf9f-c08c-4392-94e9-9cf81b3b9c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:04.635777 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0348bf9f-c08c-4392-94e9-9cf81b3b9c67 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:04.659097 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f7db8-afc4-462d-98af-da5fab52e19e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:04.662730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f7db8-afc4-462d-98af-da5fab52e19e service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:04.663072 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f7db8-afc4-462d-98af-da5fab52e19e service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:04.663470 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f7db8-afc4-462d-98af-da5fab52e19e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:04.688820 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f7db8-afc4-462d-98af-da5fab52e19e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:04.691380 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f7db8-afc4-462d-98af-da5fab52e19e service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:04.709978 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1401/2804] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:55:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:55:06.630668 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20752377-0c1d-45b4-bdf1-01f20721b411 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:06.634654 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-20752377-0c1d-45b4-bdf1-01f20721b411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:06.754632 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20752377-0c1d-45b4-bdf1-01f20721b411 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:06.759943 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bbaa5-e46c-4acd-b59c-c6e1c254c845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:06.761029 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bbaa5-e46c-4acd-b59c-c6e1c254c845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:06.761331 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bbaa5-e46c-4acd-b59c-c6e1c254c845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:06.768040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bbaa5-e46c-4acd-b59c-c6e1c254c845 admin admin] PATH_INFO: `/v3/users/cba91a03b02846298e1e1cb62b1765ea` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:06.768040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294bbaa5-e46c-4acd-b59c-c6e1c254c845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cba91a03b02846298e1e1cb62b1765ea)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:06.768040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294bbaa5-e46c-4acd-b59c-c6e1c254c845 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:07.013415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-294bbaa5-e46c-4acd-b59c-c6e1c254c845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cba91a03b02846298e1e1cb62b1765ea'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:07.014022 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-294bbaa5-e46c-4acd-b59c-c6e1c254c845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cba91a03b02846298e1e1cb62b1765ea'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:07.015175 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-294bbaa5-e46c-4acd-b59c-c6e1c254c845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cba91a03b02846298e1e1cb62b1765ea'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:07.017812 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:07.043349 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-294bbaa5-e46c-4acd-b59c-c6e1c254c845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cba91a03b02846298e1e1cb62b1765ea'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:07.054631 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-294bbaa5-e46c-4acd-b59c-c6e1c254c845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cba91a03b02846298e1e1cb62b1765ea'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:07.148345 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1404/2805] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:55:06 2022] DELETE /identity/v3/users/cba91a03b02846298e1e1cb62b1765ea => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:55:07.153346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81c847-88ab-4d6b-bfac-3a45b5c997ad None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:07.153919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81c847-88ab-4d6b-bfac-3a45b5c997ad None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:07.154086 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81c847-88ab-4d6b-bfac-3a45b5c997ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:07.173776 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-0b81c847-88ab-4d6b-bfac-3a45b5c997ad None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:07.176189 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0b81c847-88ab-4d6b-bfac-3a45b5c997ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:07.203471 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:07.264881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1402/2806] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:07.275332 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bbaa5-e46c-4acd-b59c-c6e1c254c845 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:07.277540 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-294bbaa5-e46c-4acd-b59c-c6e1c254c845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:07.381405 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bbaa5-e46c-4acd-b59c-c6e1c254c845 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:07.387302 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181d05e-420f-402e-b264-986c7df6040e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:07.388010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181d05e-420f-402e-b264-986c7df6040e service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:07.388312 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181d05e-420f-402e-b264-986c7df6040e service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:07.388654 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181d05e-420f-402e-b264-986c7df6040e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:07.401793 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181d05e-420f-402e-b264-986c7df6040e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:07.403836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181d05e-420f-402e-b264-986c7df6040e service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:07.418973 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1405/2807] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:55:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:55:07.987161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81c847-88ab-4d6b-bfac-3a45b5c997ad None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:07.989710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-0b81c847-88ab-4d6b-bfac-3a45b5c997ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:08.024118 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81c847-88ab-4d6b-bfac-3a45b5c997ad None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:08.030013 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4e445-6b6b-4deb-875c-05e031eed828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:08.032074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4e445-6b6b-4deb-875c-05e031eed828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:08.032428 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4e445-6b6b-4deb-875c-05e031eed828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:08.032784 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4e445-6b6b-4deb-875c-05e031eed828 admin admin] PATH_INFO: `/v3/projects/840cf582e6aa4391957d79242135b12f` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:08.034291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4e445-6b6b-4deb-875c-05e031eed828 admin admin] RBAC: Authorizing `identity:delete_project(project_id=840cf582e6aa4391957d79242135b12f)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:08.036567 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4e445-6b6b-4deb-875c-05e031eed828 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:08.123795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181d05e-420f-402e-b264-986c7df6040e service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:08.125437 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8181d05e-420f-402e-b264-986c7df6040e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:08.144738 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181d05e-420f-402e-b264-986c7df6040e service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:08.162058 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6da4e445-6b6b-4deb-875c-05e031eed828 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '840cf582e6aa4391957d79242135b12f', 'request_id': 'req-6da4e445-6b6b-4deb-875c-05e031eed828'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:08.180848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6da4e445-6b6b-4deb-875c-05e031eed828 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '840cf582e6aa4391957d79242135b12f', 'request_id': 'req-6da4e445-6b6b-4deb-875c-05e031eed828'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:08.190478 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6da4e445-6b6b-4deb-875c-05e031eed828 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '840cf582e6aa4391957d79242135b12f', 'request_id': 'req-6da4e445-6b6b-4deb-875c-05e031eed828'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:08.224094 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d308ba4-faa1-4328-bd5c-f702aee1db37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:08.225917 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d308ba4-faa1-4328-bd5c-f702aee1db37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:08.226429 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d308ba4-faa1-4328-bd5c-f702aee1db37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:08.226937 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d308ba4-faa1-4328-bd5c-f702aee1db37 admin admin] PATH_INFO: `/v3/users/f7ce7458121343d0a77cc49579a3d744` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:08.229417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d308ba4-faa1-4328-bd5c-f702aee1db37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7ce7458121343d0a77cc49579a3d744)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:08.234283 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d308ba4-faa1-4328-bd5c-f702aee1db37 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:08.287347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6da4e445-6b6b-4deb-875c-05e031eed828 admin admin] The token cache is being invalidate because project 840cf582e6aa4391957d79242135b12f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:55:08.287600 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6da4e445-6b6b-4deb-875c-05e031eed828 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:08.288890 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6da4e445-6b6b-4deb-875c-05e031eed828 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:08.295054 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1403/2808] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:55:07 2022] DELETE /identity/v3/projects/840cf582e6aa4391957d79242135b12f => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:55:08.296833 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4e445-6b6b-4deb-875c-05e031eed828 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:08.298631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6da4e445-6b6b-4deb-875c-05e031eed828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:08.314143 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4e445-6b6b-4deb-875c-05e031eed828 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:08.320213 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464bacd9-72b9-4a5e-8cc8-aa00feb10210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:08.321134 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bacd9-72b9-4a5e-8cc8-aa00feb10210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:08.321581 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bacd9-72b9-4a5e-8cc8-aa00feb10210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:08.322014 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bacd9-72b9-4a5e-8cc8-aa00feb10210 admin admin] PATH_INFO: `/v3/users/c7bb37d33a9c4fd489142143a2ee874b` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:08.323500 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464bacd9-72b9-4a5e-8cc8-aa00feb10210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7bb37d33a9c4fd489142143a2ee874b)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:08.326311 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464bacd9-72b9-4a5e-8cc8-aa00feb10210 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:08.514310 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3d308ba4-faa1-4328-bd5c-f702aee1db37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7ce7458121343d0a77cc49579a3d744'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:08.515883 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3d308ba4-faa1-4328-bd5c-f702aee1db37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7ce7458121343d0a77cc49579a3d744'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:08.516630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3d308ba4-faa1-4328-bd5c-f702aee1db37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7ce7458121343d0a77cc49579a3d744'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:08.543535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3d308ba4-faa1-4328-bd5c-f702aee1db37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7ce7458121343d0a77cc49579a3d744'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:08.573580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:08.576690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-464bacd9-72b9-4a5e-8cc8-aa00feb10210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7bb37d33a9c4fd489142143a2ee874b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:08.578277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-464bacd9-72b9-4a5e-8cc8-aa00feb10210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7bb37d33a9c4fd489142143a2ee874b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:08.578941 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-3d308ba4-faa1-4328-bd5c-f702aee1db37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7ce7458121343d0a77cc49579a3d744'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:08.580158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-464bacd9-72b9-4a5e-8cc8-aa00feb10210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7bb37d33a9c4fd489142143a2ee874b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:08.611717 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-464bacd9-72b9-4a5e-8cc8-aa00feb10210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7bb37d33a9c4fd489142143a2ee874b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:08.628846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-464bacd9-72b9-4a5e-8cc8-aa00feb10210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7bb37d33a9c4fd489142143a2ee874b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:08.673514 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1406/2809] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:55:08 2022] DELETE /identity/v3/users/f7ce7458121343d0a77cc49579a3d744 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:55:08.683591 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f62cf-edb1-41dd-adfa-112f24778190 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:08.687130 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f62cf-edb1-41dd-adfa-112f24778190 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:08.687130 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f62cf-edb1-41dd-adfa-112f24778190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:08.714169 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-9e5f62cf-edb1-41dd-adfa-112f24778190 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:08.715727 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9e5f62cf-edb1-41dd-adfa-112f24778190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:08.759482 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1404/2810] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:55:08 2022] DELETE /identity/v3/users/c7bb37d33a9c4fd489142143a2ee874b => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:55:08.815284 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1407/2811] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:08.831215 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464bacd9-72b9-4a5e-8cc8-aa00feb10210 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:08.833267 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-464bacd9-72b9-4a5e-8cc8-aa00feb10210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:08.893571 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:09.011797 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464bacd9-72b9-4a5e-8cc8-aa00feb10210 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:09.020566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114f324-c22b-4f33-8261-a8e77515df08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:09.021399 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114f324-c22b-4f33-8261-a8e77515df08 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:09.021872 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114f324-c22b-4f33-8261-a8e77515df08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:09.022309 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114f324-c22b-4f33-8261-a8e77515df08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:09.158009 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114f324-c22b-4f33-8261-a8e77515df08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:09.164881 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114f324-c22b-4f33-8261-a8e77515df08 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:09.180269 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1405/2812] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:55:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:55:09.882703 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f62cf-edb1-41dd-adfa-112f24778190 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:09.886171 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9e5f62cf-edb1-41dd-adfa-112f24778190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:09.918677 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f62cf-edb1-41dd-adfa-112f24778190 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:09.926346 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71398e51-9637-4e0b-8eed-2ca2b7de08d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:09.927606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71398e51-9637-4e0b-8eed-2ca2b7de08d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:09.927989 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71398e51-9637-4e0b-8eed-2ca2b7de08d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:09.928337 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71398e51-9637-4e0b-8eed-2ca2b7de08d8 admin admin] PATH_INFO: `/v3/projects/be79d6dc778640809a4a32b72fa877cc` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:09.929974 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71398e51-9637-4e0b-8eed-2ca2b7de08d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be79d6dc778640809a4a32b72fa877cc)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:09.932949 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71398e51-9637-4e0b-8eed-2ca2b7de08d8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:10.104568 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-71398e51-9637-4e0b-8eed-2ca2b7de08d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be79d6dc778640809a4a32b72fa877cc', 'request_id': 'req-71398e51-9637-4e0b-8eed-2ca2b7de08d8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:10.117208 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-71398e51-9637-4e0b-8eed-2ca2b7de08d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be79d6dc778640809a4a32b72fa877cc', 'request_id': 'req-71398e51-9637-4e0b-8eed-2ca2b7de08d8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:10.128254 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-71398e51-9637-4e0b-8eed-2ca2b7de08d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be79d6dc778640809a4a32b72fa877cc', 'request_id': 'req-71398e51-9637-4e0b-8eed-2ca2b7de08d8'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:10.247346 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114f324-c22b-4f33-8261-a8e77515df08 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:10.248816 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-71398e51-9637-4e0b-8eed-2ca2b7de08d8 admin admin] The token cache is being invalidate because project be79d6dc778640809a4a32b72fa877cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:55:10.249095 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-71398e51-9637-4e0b-8eed-2ca2b7de08d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:10.250127 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-71398e51-9637-4e0b-8eed-2ca2b7de08d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:10.259147 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1408/2813] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:55:09 2022] DELETE /identity/v3/projects/be79d6dc778640809a4a32b72fa877cc => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:55:10.260413 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71398e51-9637-4e0b-8eed-2ca2b7de08d8 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:10.260637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-4114f324-c22b-4f33-8261-a8e77515df08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:10.263951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-71398e51-9637-4e0b-8eed-2ca2b7de08d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:10.293854 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114f324-c22b-4f33-8261-a8e77515df08 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:10.423209 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb7a4e-ffc8-4b2a-8baa-93689129818b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:10.424081 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb7a4e-ffc8-4b2a-8baa-93689129818b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:10.424342 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb7a4e-ffc8-4b2a-8baa-93689129818b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:10.424631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb7a4e-ffc8-4b2a-8baa-93689129818b admin admin] PATH_INFO: `/v3/projects/c75ce285523c420a89891d9520d9595e` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:10.426499 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb7a4e-ffc8-4b2a-8baa-93689129818b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c75ce285523c420a89891d9520d9595e)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:10.429001 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb7a4e-ffc8-4b2a-8baa-93689129818b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:10.470829 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71398e51-9637-4e0b-8eed-2ca2b7de08d8 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:10.479587 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd89819-98aa-45f6-91af-6347dcf63879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:10.481234 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd89819-98aa-45f6-91af-6347dcf63879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:10.481847 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd89819-98aa-45f6-91af-6347dcf63879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:10.482485 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd89819-98aa-45f6-91af-6347dcf63879 admin admin] PATH_INFO: `/v3/users/09438fcf496146caba8b681fb1c2ae4b` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:10.486343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd89819-98aa-45f6-91af-6347dcf63879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09438fcf496146caba8b681fb1c2ae4b)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:10.490655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd89819-98aa-45f6-91af-6347dcf63879 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:10.520943 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:10.633703 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-fdfb7a4e-ffc8-4b2a-8baa-93689129818b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c75ce285523c420a89891d9520d9595e', 'request_id': 'req-fdfb7a4e-ffc8-4b2a-8baa-93689129818b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:10.647552 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-fdfb7a4e-ffc8-4b2a-8baa-93689129818b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c75ce285523c420a89891d9520d9595e', 'request_id': 'req-fdfb7a4e-ffc8-4b2a-8baa-93689129818b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:10.661814 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-fdfb7a4e-ffc8-4b2a-8baa-93689129818b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c75ce285523c420a89891d9520d9595e', 'request_id': 'req-fdfb7a4e-ffc8-4b2a-8baa-93689129818b'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:10.752298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2dd89819-98aa-45f6-91af-6347dcf63879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09438fcf496146caba8b681fb1c2ae4b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:10.754516 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2dd89819-98aa-45f6-91af-6347dcf63879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09438fcf496146caba8b681fb1c2ae4b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:10.756160 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2dd89819-98aa-45f6-91af-6347dcf63879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09438fcf496146caba8b681fb1c2ae4b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:10.772594 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:10.784552 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-fdfb7a4e-ffc8-4b2a-8baa-93689129818b admin admin] The token cache is being invalidate because project c75ce285523c420a89891d9520d9595e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:55:10.785088 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-fdfb7a4e-ffc8-4b2a-8baa-93689129818b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:10.786323 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-fdfb7a4e-ffc8-4b2a-8baa-93689129818b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:10.794462 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1406/2814] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:55:10 2022] DELETE /identity/v3/projects/c75ce285523c420a89891d9520d9595e => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:55:10.797649 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2dd89819-98aa-45f6-91af-6347dcf63879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09438fcf496146caba8b681fb1c2ae4b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:10.800742 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed89d7-e8f9-4a9d-9483-cf123ac95879 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:10.801523 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed89d7-e8f9-4a9d-9483-cf123ac95879 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:10.802730 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed89d7-e8f9-4a9d-9483-cf123ac95879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:10.828248 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-2dd89819-98aa-45f6-91af-6347dcf63879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09438fcf496146caba8b681fb1c2ae4b'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:10.846297 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-e0ed89d7-e8f9-4a9d-9483-cf123ac95879 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:10.848791 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e0ed89d7-e8f9-4a9d-9483-cf123ac95879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:10.917594 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1409/2815] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:55:10 2022] DELETE /identity/v3/users/09438fcf496146caba8b681fb1c2ae4b => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:55:10.997697 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1407/2816] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:10.998868 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879e6ac-6465-422a-b23e-bb9f4feac1c7 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:10.999392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879e6ac-6465-422a-b23e-bb9f4feac1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:10.999852 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879e6ac-6465-422a-b23e-bb9f4feac1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:11.027759 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-e879e6ac-6465-422a-b23e-bb9f4feac1c7 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:11.028961 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e879e6ac-6465-422a-b23e-bb9f4feac1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:11.134700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1410/2817] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:11.137282 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed89d7-e8f9-4a9d-9483-cf123ac95879 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:11.152948 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed89d7-e8f9-4a9d-9483-cf123ac95879 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:11.164187 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebc8d4-d820-4972-a28a-a756870b83bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:11.165866 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebc8d4-d820-4972-a28a-a756870b83bd admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:11.166574 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebc8d4-d820-4972-a28a-a756870b83bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:11.167168 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebc8d4-d820-4972-a28a-a756870b83bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:11.168734 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebc8d4-d820-4972-a28a-a756870b83bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:55:11.169459 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebc8d4-d820-4972-a28a-a756870b83bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:11.173112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebc8d4-d820-4972-a28a-a756870b83bd admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:11.206431 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1408/2818] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:55:11.215731 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879e6ac-6465-422a-b23e-bb9f4feac1c7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:11.266513 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879e6ac-6465-422a-b23e-bb9f4feac1c7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:11.277975 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e2336-1814-4259-8745-dc489db6a45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:11.279332 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e2336-1814-4259-8745-dc489db6a45f admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:11.280026 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e2336-1814-4259-8745-dc489db6a45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:11.280610 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e2336-1814-4259-8745-dc489db6a45f admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:11.281666 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e2336-1814-4259-8745-dc489db6a45f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:11.285383 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e2336-1814-4259-8745-dc489db6a45f admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:11.432542 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1411/2819] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:55:11 2022] POST /identity/v3/projects => generated 381 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:55:11.433339 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebc8d4-d820-4972-a28a-a756870b83bd admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:11.491243 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebc8d4-d820-4972-a28a-a756870b83bd admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:11.560155 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e2336-1814-4259-8745-dc489db6a45f admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:11.562183 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-117e2336-1814-4259-8745-dc489db6a45f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:11.640138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01344952-3953-49fa-a0f2-3f786b90ebdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:11.641709 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01344952-3953-49fa-a0f2-3f786b90ebdb admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:11.641996 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01344952-3953-49fa-a0f2-3f786b90ebdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:11.642286 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01344952-3953-49fa-a0f2-3f786b90ebdb admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:11.643421 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01344952-3953-49fa-a0f2-3f786b90ebdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:11.646796 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01344952-3953-49fa-a0f2-3f786b90ebdb admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:11.696789 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e2336-1814-4259-8745-dc489db6a45f admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:11.703725 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fbfeac-0a92-4aac-9f27-c21d737b1c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:11.704782 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbfeac-0a92-4aac-9f27-c21d737b1c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:11.705159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbfeac-0a92-4aac-9f27-c21d737b1c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:11.705593 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbfeac-0a92-4aac-9f27-c21d737b1c26 admin admin] PATH_INFO: `/v3/projects/acc58895b5de45bfa9b683b76b3a3f90` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:11.707092 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fbfeac-0a92-4aac-9f27-c21d737b1c26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acc58895b5de45bfa9b683b76b3a3f90)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:11.709424 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fbfeac-0a92-4aac-9f27-c21d737b1c26 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:11.731247 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1409/2820] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:55:11 2022] POST /identity/v3/users => generated 291 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:55:11.732613 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01344952-3953-49fa-a0f2-3f786b90ebdb admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:11.736277 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:11.756529 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01344952-3953-49fa-a0f2-3f786b90ebdb admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:11.766251 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba753507-7d83-4e7c-ab47-400d801914e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:11.767464 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba753507-7d83-4e7c-ab47-400d801914e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:11.768016 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba753507-7d83-4e7c-ab47-400d801914e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:11.768689 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba753507-7d83-4e7c-ab47-400d801914e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:11.770136 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba753507-7d83-4e7c-ab47-400d801914e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:11.773235 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba753507-7d83-4e7c-ab47-400d801914e5 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:11.814023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1410/2821] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:55:11.818835 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba753507-7d83-4e7c-ab47-400d801914e5 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:11.855628 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba753507-7d83-4e7c-ab47-400d801914e5 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:11.866126 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-92fbfeac-0a92-4aac-9f27-c21d737b1c26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acc58895b5de45bfa9b683b76b3a3f90', 'request_id': 'req-92fbfeac-0a92-4aac-9f27-c21d737b1c26'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:11.878714 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-92fbfeac-0a92-4aac-9f27-c21d737b1c26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acc58895b5de45bfa9b683b76b3a3f90', 'request_id': 'req-92fbfeac-0a92-4aac-9f27-c21d737b1c26'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:11.901040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-92fbfeac-0a92-4aac-9f27-c21d737b1c26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acc58895b5de45bfa9b683b76b3a3f90', 'request_id': 'req-92fbfeac-0a92-4aac-9f27-c21d737b1c26'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:11.976642 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77861ec5-27c3-42d1-b6df-80b1e3832a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:11.977599 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77861ec5-27c3-42d1-b6df-80b1e3832a17 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:11.977792 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77861ec5-27c3-42d1-b6df-80b1e3832a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:11.977988 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77861ec5-27c3-42d1-b6df-80b1e3832a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:11.978987 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77861ec5-27c3-42d1-b6df-80b1e3832a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:11.980922 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77861ec5-27c3-42d1-b6df-80b1e3832a17 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:11.991190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:11.997598 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1411/2822] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:55:12.000584 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-92fbfeac-0a92-4aac-9f27-c21d737b1c26 admin admin] The token cache is being invalidate because project acc58895b5de45bfa9b683b76b3a3f90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:55:12.002255 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-92fbfeac-0a92-4aac-9f27-c21d737b1c26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:12.003138 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77861ec5-27c3-42d1-b6df-80b1e3832a17 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:12.004143 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-92fbfeac-0a92-4aac-9f27-c21d737b1c26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:12.005403 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-77861ec5-27c3-42d1-b6df-80b1e3832a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:12.008242 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1412/2823] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:55:11 2022] DELETE /identity/v3/projects/acc58895b5de45bfa9b683b76b3a3f90 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:55:12.019168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93a165-a298-4c6b-9a0c-2482b2298b70 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:12.020001 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93a165-a298-4c6b-9a0c-2482b2298b70 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:12.021069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93a165-a298-4c6b-9a0c-2482b2298b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:12.039453 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77861ec5-27c3-42d1-b6df-80b1e3832a17 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:12.065710 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b49cdc-7ea6-4bd0-834e-ad32520e59c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:12.068503 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b49cdc-7ea6-4bd0-834e-ad32520e59c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:12.069151 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-7f93a165-a298-4c6b-9a0c-2482b2298b70 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:12.069320 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b49cdc-7ea6-4bd0-834e-ad32520e59c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:12.069893 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b49cdc-7ea6-4bd0-834e-ad32520e59c1 admin admin] PATH_INFO: `/v3/projects/3df75aa755784fee814abfd669ce02da/users/b9f08551f4ff4bbd9c48c438a518c8d0/roles/587af95739954600bbf2411974601f74` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:12.070425 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-7f93a165-a298-4c6b-9a0c-2482b2298b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:12.093286 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1413/2824] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:55:12.102515 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbfd13-e40a-4d4b-9c62-c591414891e9 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:12.103104 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbfd13-e40a-4d4b-9c62-c591414891e9 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:12.103553 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbfd13-e40a-4d4b-9c62-c591414891e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:12.126020 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b49cdc-7ea6-4bd0-834e-ad32520e59c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df75aa755784fee814abfd669ce02da, user_id=b9f08551f4ff4bbd9c48c438a518c8d0, role_id=587af95739954600bbf2411974601f74)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:12.131279 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-a0bbfd13-e40a-4d4b-9c62-c591414891e9 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:12.132655 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a0bbfd13-e40a-4d4b-9c62-c591414891e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:12.134023 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b49cdc-7ea6-4bd0-834e-ad32520e59c1 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:12.142586 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1414/2825] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:12.147168 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbfd13-e40a-4d4b-9c62-c591414891e9 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:12.158631 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbfd13-e40a-4d4b-9c62-c591414891e9 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:12.164882 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee17e2-968a-430a-b190-289f484680f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:12.165861 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee17e2-968a-430a-b190-289f484680f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:12.166264 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee17e2-968a-430a-b190-289f484680f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:12.166634 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee17e2-968a-430a-b190-289f484680f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:12.167523 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee17e2-968a-430a-b190-289f484680f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80339) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:55:12.167963 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee17e2-968a-430a-b190-289f484680f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:12.169697 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee17e2-968a-430a-b190-289f484680f9 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:12.185643 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1415/2826] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 08:55:12.190525 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee17e2-968a-430a-b190-289f484680f9 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:12.204407 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee17e2-968a-430a-b190-289f484680f9 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:12.213173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcd117-b1ec-468b-a05f-19654b3b1e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:12.214089 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcd117-b1ec-468b-a05f-19654b3b1e42 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:12.214418 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcd117-b1ec-468b-a05f-19654b3b1e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:12.214885 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcd117-b1ec-468b-a05f-19654b3b1e42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:12.215612 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bcd117-b1ec-468b-a05f-19654b3b1e42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:12.218667 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bcd117-b1ec-468b-a05f-19654b3b1e42 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:12.229052 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1412/2827] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:55:12 2022] PUT /identity/v3/projects/3df75aa755784fee814abfd669ce02da/users/b9f08551f4ff4bbd9c48c438a518c8d0/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:55:12.235234 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b155940-746e-4b79-872c-ce99a3594340 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:12.237968 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b155940-746e-4b79-872c-ce99a3594340 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:12.237968 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b155940-746e-4b79-872c-ce99a3594340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:12.261339 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-2b155940-746e-4b79-872c-ce99a3594340 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:12.262834 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2b155940-746e-4b79-872c-ce99a3594340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:12.351901 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1416/2828] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:55:12 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 08:55:12.356456 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcd117-b1ec-468b-a05f-19654b3b1e42 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:12.394677 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1413/2829] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:55:12.404757 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b155940-746e-4b79-872c-ce99a3594340 None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:12.409356 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-2b155940-746e-4b79-872c-ce99a3594340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:12.472608 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcd117-b1ec-468b-a05f-19654b3b1e42 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:12.484767 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7461f7-677e-47b8-92c6-65e889a3b168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:12.486640 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7461f7-677e-47b8-92c6-65e889a3b168 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:12.487292 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7461f7-677e-47b8-92c6-65e889a3b168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:12.488212 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7461f7-677e-47b8-92c6-65e889a3b168 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:12.489909 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7461f7-677e-47b8-92c6-65e889a3b168 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:12.502071 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b155940-746e-4b79-872c-ce99a3594340 None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:12.504146 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7461f7-677e-47b8-92c6-65e889a3b168 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:12.571291 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce9de0b-ad88-4962-b5ab-b0b1137f21d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:12.572078 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9de0b-ad88-4962-b5ab-b0b1137f21d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:12.572385 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9de0b-ad88-4962-b5ab-b0b1137f21d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:12.572656 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9de0b-ad88-4962-b5ab-b0b1137f21d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:12.589158 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce9de0b-ad88-4962-b5ab-b0b1137f21d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:12.590811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce9de0b-ad88-4962-b5ab-b0b1137f21d2 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:12.602004 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1417/2830] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:55:12 2022] POST /identity/v3/users => generated 287 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:55:12.607069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7461f7-677e-47b8-92c6-65e889a3b168 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:12.625190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1414/2831] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:55:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:55:12.627739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7461f7-677e-47b8-92c6-65e889a3b168 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:12.633157 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad016fee-a4d8-4dd5-9b09-7fadb4c720a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:12.634000 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad016fee-a4d8-4dd5-9b09-7fadb4c720a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:12.634353 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad016fee-a4d8-4dd5-9b09-7fadb4c720a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:12.634738 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad016fee-a4d8-4dd5-9b09-7fadb4c720a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:12.635570 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad016fee-a4d8-4dd5-9b09-7fadb4c720a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:12.637615 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad016fee-a4d8-4dd5-9b09-7fadb4c720a2 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:12.663454 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1418/2832] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:55:12.669017 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce9de0b-ad88-4962-b5ab-b0b1137f21d2 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:12.686809 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce9de0b-ad88-4962-b5ab-b0b1137f21d2 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:12.696098 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ee03b-b2b5-4fcf-a81b-99fb1d56d0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:12.697997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ee03b-b2b5-4fcf-a81b-99fb1d56d0ac admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:12.697997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ee03b-b2b5-4fcf-a81b-99fb1d56d0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:12.697997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ee03b-b2b5-4fcf-a81b-99fb1d56d0ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:12.703819 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ee03b-b2b5-4fcf-a81b-99fb1d56d0ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:12.708997 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ee03b-b2b5-4fcf-a81b-99fb1d56d0ac admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:12.742189 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1415/2833] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:55:12.744107 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad016fee-a4d8-4dd5-9b09-7fadb4c720a2 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:12.776424 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad016fee-a4d8-4dd5-9b09-7fadb4c720a2 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:12.794490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf5dcc-76fa-427e-b00f-319cf52599f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:12.796228 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf5dcc-76fa-427e-b00f-319cf52599f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:12.796981 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf5dcc-76fa-427e-b00f-319cf52599f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:12.797483 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf5dcc-76fa-427e-b00f-319cf52599f4 admin admin] PATH_INFO: `/v3/projects/98f69056ca3e4e7c83f6c36c8e16c16c/users/124957514a4e4c35bed50a19de1dda6d/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:12.850332 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf5dcc-76fa-427e-b00f-319cf52599f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98f69056ca3e4e7c83f6c36c8e16c16c, user_id=124957514a4e4c35bed50a19de1dda6d, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:12.869541 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf5dcc-76fa-427e-b00f-319cf52599f4 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:12.956427 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1419/2834] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:55:12 2022] PUT /identity/v3/projects/98f69056ca3e4e7c83f6c36c8e16c16c/users/124957514a4e4c35bed50a19de1dda6d/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 08:55:12.964014 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc02b0d-a2de-4cc9-9367-fa1552a40049 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:12.964418 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc02b0d-a2de-4cc9-9367-fa1552a40049 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:12.965002 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc02b0d-a2de-4cc9-9367-fa1552a40049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:12.989378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-8fc02b0d-a2de-4cc9-9367-fa1552a40049 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:12.990676 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-8fc02b0d-a2de-4cc9-9367-fa1552a40049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:13.019636 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:13.135513 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1416/2835] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:13.150114 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf5dcc-76fa-427e-b00f-319cf52599f4 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:13.168735 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:13.244536 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf5dcc-76fa-427e-b00f-319cf52599f4 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:13.249752 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cfa4c9-e58b-4e58-94fa-ca0a7a6c451c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:13.250401 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfa4c9-e58b-4e58-94fa-ca0a7a6c451c service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:13.250709 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfa4c9-e58b-4e58-94fa-ca0a7a6c451c service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:13.251030 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfa4c9-e58b-4e58-94fa-ca0a7a6c451c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:13.262076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cfa4c9-e58b-4e58-94fa-ca0a7a6c451c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:13.263804 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cfa4c9-e58b-4e58-94fa-ca0a7a6c451c service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:13.283388 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1420/2836] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:55:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:55:14.730792 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273bb0f-d521-4eca-80cf-0cbac1f04caf None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:14.731640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273bb0f-d521-4eca-80cf-0cbac1f04caf None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:14.732051 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273bb0f-d521-4eca-80cf-0cbac1f04caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:14.782394 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-1273bb0f-d521-4eca-80cf-0cbac1f04caf None None] MFA Rules not processed for user `b9f08551f4ff4bbd9c48c438a518c8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:14.784664 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1273bb0f-d521-4eca-80cf-0cbac1f04caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:14.877690 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1417/2837] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:14.888947 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cfa4c9-e58b-4e58-94fa-ca0a7a6c451c service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:14.891578 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d7cfa4c9-e58b-4e58-94fa-ca0a7a6c451c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:14.971478 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cfa4c9-e58b-4e58-94fa-ca0a7a6c451c service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:14.975887 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe08e3-6034-486e-9ae2-ed616615e2e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:14.976595 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe08e3-6034-486e-9ae2-ed616615e2e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:14.976873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe08e3-6034-486e-9ae2-ed616615e2e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:14.977148 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe08e3-6034-486e-9ae2-ed616615e2e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:14.988495 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe08e3-6034-486e-9ae2-ed616615e2e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:14.990037 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe08e3-6034-486e-9ae2-ed616615e2e5 service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:15.000873 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1421/2838] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:55:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:55:15.275458 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273bb0f-d521-4eca-80cf-0cbac1f04caf None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:15.277877 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1273bb0f-d521-4eca-80cf-0cbac1f04caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:15.300728 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273bb0f-d521-4eca-80cf-0cbac1f04caf None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:15.307721 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a88ce6-4a44-4074-850f-67da79c8563b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:15.308873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a88ce6-4a44-4074-850f-67da79c8563b service cinder] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:15.309384 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a88ce6-4a44-4074-850f-67da79c8563b service cinder] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:15.309919 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a88ce6-4a44-4074-850f-67da79c8563b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:15.316474 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-45a88ce6-4a44-4074-850f-67da79c8563b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:15.424569 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a88ce6-4a44-4074-850f-67da79c8563b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:15.428277 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a88ce6-4a44-4074-850f-67da79c8563b service cinder] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:15.445467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1418/2839] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:55:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:55:16.395801 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df2451-43d2-4dd4-89c6-0db44f663876 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:16.397896 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df2451-43d2-4dd4-89c6-0db44f663876 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:16.398405 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df2451-43d2-4dd4-89c6-0db44f663876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:16.431476 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-73df2451-43d2-4dd4-89c6-0db44f663876 None None] MFA Rules not processed for user `2b03f8b8c51e472bbd4cf0afc9c6b632`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:16.434859 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-73df2451-43d2-4dd4-89c6-0db44f663876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:16.600320 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1422/2840] 10.222.0.57 () {58 vars in 967 bytes} [Mon Jul 18 08:55:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:16.618268 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a88ce6-4a44-4074-850f-67da79c8563b service cinder] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:16.620622 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-45a88ce6-4a44-4074-850f-67da79c8563b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:16.707998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=100 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:16.753193 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a88ce6-4a44-4074-850f-67da79c8563b service cinder] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:16.766701 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7b005-ff5c-481c-a7f5-f3f5eb6ca7b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09a5c2e804504038817cad76544dce5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:16.767872 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7b005-ff5c-481c-a7f5-f3f5eb6ca7b6 service swift] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:16.769454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7b005-ff5c-481c-a7f5-f3f5eb6ca7b6 service swift] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:16.770744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7b005-ff5c-481c-a7f5-f3f5eb6ca7b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:16.784759 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7b005-ff5c-481c-a7f5-f3f5eb6ca7b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:16.786474 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7b005-ff5c-481c-a7f5-f3f5eb6ca7b6 service swift] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:16.798972 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1419/2841] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Jul 18 08:55:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 08:55:16.935326 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949414df-a13b-4ae4-9cbe-2e193e03955c None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:16.935625 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949414df-a13b-4ae4-9cbe-2e193e03955c None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:16.935955 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949414df-a13b-4ae4-9cbe-2e193e03955c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:16.964295 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-949414df-a13b-4ae4-9cbe-2e193e03955c None None] MFA Rules not processed for user `b9f08551f4ff4bbd9c48c438a518c8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:16.965893 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-949414df-a13b-4ae4-9cbe-2e193e03955c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:16.978695 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1423/2842] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:55:21.442937 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7b005-ff5c-481c-a7f5-f3f5eb6ca7b6 service swift] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:21.446489 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-fdf7b005-ff5c-481c-a7f5-f3f5eb6ca7b6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:21.473033 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7b005-ff5c-481c-a7f5-f3f5eb6ca7b6 service swift] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:21.483134 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4c31a-9bc8-40ff-843e-ff83844ebbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:21.484721 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4c31a-9bc8-40ff-843e-ff83844ebbb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:21.485232 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4c31a-9bc8-40ff-843e-ff83844ebbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:21.485743 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4c31a-9bc8-40ff-843e-ff83844ebbb4 admin admin] PATH_INFO: `/v3/users/4e8cc08181c74bb584fdca44adba662a` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:21.488782 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4c31a-9bc8-40ff-843e-ff83844ebbb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e8cc08181c74bb584fdca44adba662a)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:21.493455 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4c31a-9bc8-40ff-843e-ff83844ebbb4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:21.790018 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-82a4c31a-9bc8-40ff-843e-ff83844ebbb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e8cc08181c74bb584fdca44adba662a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:21.791197 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-82a4c31a-9bc8-40ff-843e-ff83844ebbb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e8cc08181c74bb584fdca44adba662a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:21.792472 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-82a4c31a-9bc8-40ff-843e-ff83844ebbb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e8cc08181c74bb584fdca44adba662a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:21.823022 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-82a4c31a-9bc8-40ff-843e-ff83844ebbb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e8cc08181c74bb584fdca44adba662a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:21.841045 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-82a4c31a-9bc8-40ff-843e-ff83844ebbb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e8cc08181c74bb584fdca44adba662a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:21.999266 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1420/2843] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:55:21 2022] DELETE /identity/v3/users/4e8cc08181c74bb584fdca44adba662a => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:55:22.421395 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba9e02-342f-41ec-b3b3-e3c0dcc6180f None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:22.422215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba9e02-342f-41ec-b3b3-e3c0dcc6180f None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:22.422215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba9e02-342f-41ec-b3b3-e3c0dcc6180f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:22.461749 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-d2ba9e02-342f-41ec-b3b3-e3c0dcc6180f None None] MFA Rules not processed for user `124957514a4e4c35bed50a19de1dda6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:22.463359 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d2ba9e02-342f-41ec-b3b3-e3c0dcc6180f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:22.517170 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4c31a-9bc8-40ff-843e-ff83844ebbb4 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:22.522112 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-82a4c31a-9bc8-40ff-843e-ff83844ebbb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:22.566700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:22.630978 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1424/2844] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:55:22.632535 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba9e02-342f-41ec-b3b3-e3c0dcc6180f None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:22.634368 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-d2ba9e02-342f-41ec-b3b3-e3c0dcc6180f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:22.739901 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4c31a-9bc8-40ff-843e-ff83844ebbb4 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:22.747265 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2254f3-b46c-4c0d-8b2a-a5cba833a3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:22.748400 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2254f3-b46c-4c0d-8b2a-a5cba833a3e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:22.749485 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba9e02-342f-41ec-b3b3-e3c0dcc6180f None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:22.749775 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2254f3-b46c-4c0d-8b2a-a5cba833a3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:22.750433 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2254f3-b46c-4c0d-8b2a-a5cba833a3e7 admin admin] PATH_INFO: `/v3/projects/401f21125b3146fe9deba3df1abf1803` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:22.753774 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2254f3-b46c-4c0d-8b2a-a5cba833a3e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=401f21125b3146fe9deba3df1abf1803)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:22.755309 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46cc6d-4a64-434c-a752-3bca12dfb011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:22.756129 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46cc6d-4a64-434c-a752-3bca12dfb011 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:22.756334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46cc6d-4a64-434c-a752-3bca12dfb011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:22.756618 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46cc6d-4a64-434c-a752-3bca12dfb011 admin admin] PATH_INFO: `/v3/projects` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:22.757347 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46cc6d-4a64-434c-a752-3bca12dfb011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:22.763561 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46cc6d-4a64-434c-a752-3bca12dfb011 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:22.764554 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2254f3-b46c-4c0d-8b2a-a5cba833a3e7 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:22.873385 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1425/2845] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:55:22 2022] POST /identity/v3/projects => generated 371 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:55:22.877528 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46cc6d-4a64-434c-a752-3bca12dfb011 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:22.950190 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6d2254f3-b46c-4c0d-8b2a-a5cba833a3e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '401f21125b3146fe9deba3df1abf1803', 'request_id': 'req-6d2254f3-b46c-4c0d-8b2a-a5cba833a3e7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:22.966047 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:22.969974 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6d2254f3-b46c-4c0d-8b2a-a5cba833a3e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '401f21125b3146fe9deba3df1abf1803', 'request_id': 'req-6d2254f3-b46c-4c0d-8b2a-a5cba833a3e7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:23.002731 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6d2254f3-b46c-4c0d-8b2a-a5cba833a3e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '401f21125b3146fe9deba3df1abf1803', 'request_id': 'req-6d2254f3-b46c-4c0d-8b2a-a5cba833a3e7'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:23.002853 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46cc6d-4a64-434c-a752-3bca12dfb011 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:23.132500 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddcc1e-be2b-4002-b2ad-ed1c345face6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:23.147471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddcc1e-be2b-4002-b2ad-ed1c345face6 admin admin] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:23.147471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddcc1e-be2b-4002-b2ad-ed1c345face6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:23.147471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddcc1e-be2b-4002-b2ad-ed1c345face6 admin admin] PATH_INFO: `/v3/users` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:23.147471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddcc1e-be2b-4002-b2ad-ed1c345face6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:23.147471 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddcc1e-be2b-4002-b2ad-ed1c345face6 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:23.214607 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6d2254f3-b46c-4c0d-8b2a-a5cba833a3e7 admin admin] The token cache is being invalidate because project 401f21125b3146fe9deba3df1abf1803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:55:23.215303 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6d2254f3-b46c-4c0d-8b2a-a5cba833a3e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:23.216641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-6d2254f3-b46c-4c0d-8b2a-a5cba833a3e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:23.227720 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1421/2846] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:55:22 2022] DELETE /identity/v3/projects/401f21125b3146fe9deba3df1abf1803 => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:55:23.233024 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58441a-44b4-4fe7-95a4-c4445976884c None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:23.233756 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58441a-44b4-4fe7-95a4-c4445976884c None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:23.234514 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58441a-44b4-4fe7-95a4-c4445976884c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:23.269113 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-bf58441a-44b4-4fe7-95a4-c4445976884c None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:23.271918 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-bf58441a-44b4-4fe7-95a4-c4445976884c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:23.293544 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1422/2847] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:23.383318 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211f297-d316-4454-aa88-3769f7a0a3fc None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:23.383640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211f297-d316-4454-aa88-3769f7a0a3fc None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:23.383921 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211f297-d316-4454-aa88-3769f7a0a3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:23.404355 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d211f297-d316-4454-aa88-3769f7a0a3fc None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:23.405819 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d211f297-d316-4454-aa88-3769f7a0a3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:23.416279 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1423/2848] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:55:23.418811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211f297-d316-4454-aa88-3769f7a0a3fc None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:23.429892 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211f297-d316-4454-aa88-3769f7a0a3fc None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:23.440669 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4743e21e-f224-44c5-8e08-325c2531ae67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:23.441506 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743e21e-f224-44c5-8e08-325c2531ae67 admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:23.442014 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743e21e-f224-44c5-8e08-325c2531ae67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:23.442440 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743e21e-f224-44c5-8e08-325c2531ae67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:23.443258 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4743e21e-f224-44c5-8e08-325c2531ae67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 08:55:23.443614 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4743e21e-f224-44c5-8e08-325c2531ae67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:23.449745 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4743e21e-f224-44c5-8e08-325c2531ae67 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:23.459159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1426/2849] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:55:22 2022] POST /identity/v3/users => generated 292 bytes in 582 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:55:23.461137 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddcc1e-be2b-4002-b2ad-ed1c345face6 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:23.464732 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9eddcc1e-be2b-4002-b2ad-ed1c345face6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:23.472542 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1424/2850] 10.222.0.57 () {58 vars in 1152 bytes} [Mon Jul 18 08:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 08:55:23.473741 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4743e21e-f224-44c5-8e08-325c2531ae67 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:23.474801 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:23.481334 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddcc1e-be2b-4002-b2ad-ed1c345face6 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:23.488995 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75486339-3e11-4827-98f5-558db0609f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:23.496035 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4743e21e-f224-44c5-8e08-325c2531ae67 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:23.496235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75486339-3e11-4827-98f5-558db0609f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:23.496235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75486339-3e11-4827-98f5-558db0609f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:23.496235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75486339-3e11-4827-98f5-558db0609f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:23.496235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75486339-3e11-4827-98f5-558db0609f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:23.496235 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75486339-3e11-4827-98f5-558db0609f92 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:23.500999 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecaa41-c30a-445d-b263-75c94e61b111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:23.502247 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecaa41-c30a-445d-b263-75c94e61b111 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:23.503276 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecaa41-c30a-445d-b263-75c94e61b111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:23.504330 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecaa41-c30a-445d-b263-75c94e61b111 admin admin] PATH_INFO: `/v3/projects` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:23.508549 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ecaa41-c30a-445d-b263-75c94e61b111 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:23.510618 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ecaa41-c30a-445d-b263-75c94e61b111 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:23.526398 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1427/2851] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:55:23.528446 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75486339-3e11-4827-98f5-558db0609f92 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:23.554861 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75486339-3e11-4827-98f5-558db0609f92 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:23.564204 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b1006-d141-453d-bda2-26a00d1b98d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:23.565792 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b1006-d141-453d-bda2-26a00d1b98d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:23.566268 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b1006-d141-453d-bda2-26a00d1b98d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:23.566918 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b1006-d141-453d-bda2-26a00d1b98d1 admin admin] PATH_INFO: `/v3/projects/392f7faedc094ec5aaf10f1961fa5300/users/4449dce2c8b14df48b8746548e5ed55a/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:23.591159 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b1006-d141-453d-bda2-26a00d1b98d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=392f7faedc094ec5aaf10f1961fa5300, user_id=4449dce2c8b14df48b8746548e5ed55a, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:23.596390 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b1006-d141-453d-bda2-26a00d1b98d1 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:23.715274 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1425/2852] 10.222.0.57 () {60 vars in 1152 bytes} [Mon Jul 18 08:55:23 2022] POST /identity/v3/projects => generated 375 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:55:23.719925 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecaa41-c30a-445d-b263-75c94e61b111 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:23.753127 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1428/2853] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:55:23 2022] PUT /identity/v3/projects/392f7faedc094ec5aaf10f1961fa5300/users/4449dce2c8b14df48b8746548e5ed55a/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:55:23.762904 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b1006-d141-453d-bda2-26a00d1b98d1 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:23.839964 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecaa41-c30a-445d-b263-75c94e61b111 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:23.867142 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b1006-d141-453d-bda2-26a00d1b98d1 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:23.872853 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474d192-c0f6-482b-8640-41457a43546c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:23.873603 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474d192-c0f6-482b-8640-41457a43546c admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:23.874024 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474d192-c0f6-482b-8640-41457a43546c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:23.875055 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474d192-c0f6-482b-8640-41457a43546c admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:23.875886 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7474d192-c0f6-482b-8640-41457a43546c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:23.878457 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7474d192-c0f6-482b-8640-41457a43546c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:23.897149 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1429/2854] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 08:55:23.898357 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474d192-c0f6-482b-8640-41457a43546c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:23.912411 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474d192-c0f6-482b-8640-41457a43546c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:23.919332 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bfaf1-0dfb-435b-b7d7-ca6443339f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:23.922470 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bfaf1-0dfb-435b-b7d7-ca6443339f86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:23.922768 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bfaf1-0dfb-435b-b7d7-ca6443339f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:23.923053 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bfaf1-0dfb-435b-b7d7-ca6443339f86 admin admin] PATH_INFO: `/v3/projects/392f7faedc094ec5aaf10f1961fa5300/users/4449dce2c8b14df48b8746548e5ed55a/roles/467d49b8a0d3435aaba58464cc8ef85c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:23.925316 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55bfaf1-0dfb-435b-b7d7-ca6443339f86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=392f7faedc094ec5aaf10f1961fa5300, user_id=4449dce2c8b14df48b8746548e5ed55a, role_id=467d49b8a0d3435aaba58464cc8ef85c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:23.928062 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55bfaf1-0dfb-435b-b7d7-ca6443339f86 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:23.941734 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40312480-b79b-482a-91dd-c385b299a336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:23.942693 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40312480-b79b-482a-91dd-c385b299a336 admin admin] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:23.943144 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40312480-b79b-482a-91dd-c385b299a336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:23.943611 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40312480-b79b-482a-91dd-c385b299a336 admin admin] PATH_INFO: `/v3/users` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:23.944515 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40312480-b79b-482a-91dd-c385b299a336 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:23.947127 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40312480-b79b-482a-91dd-c385b299a336 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:23.959076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1430/2855] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:55:23 2022] PUT /identity/v3/projects/392f7faedc094ec5aaf10f1961fa5300/users/4449dce2c8b14df48b8746548e5ed55a/roles/467d49b8a0d3435aaba58464cc8ef85c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:55:24.140862 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1426/2856] 10.222.0.57 () {60 vars in 1143 bytes} [Mon Jul 18 08:55:23 2022] POST /identity/v3/users => generated 288 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 08:55:24.147573 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bfaf1-0dfb-435b-b7d7-ca6443339f86 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:24.258193 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bfaf1-0dfb-435b-b7d7-ca6443339f86 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:24.268043 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed12a0-3978-4384-b5fc-ecab3e0add2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:24.269570 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed12a0-3978-4384-b5fc-ecab3e0add2d admin admin] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:24.270073 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed12a0-3978-4384-b5fc-ecab3e0add2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:24.270653 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed12a0-3978-4384-b5fc-ecab3e0add2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:24.281596 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed12a0-3978-4384-b5fc-ecab3e0add2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:24.283417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed12a0-3978-4384-b5fc-ecab3e0add2d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:24.313094 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1431/2857] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:55:24.317992 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40312480-b79b-482a-91dd-c385b299a336 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:24.361407 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40312480-b79b-482a-91dd-c385b299a336 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:24.371128 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3b4ee-3b7a-4c76-a319-191b80de6d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:24.373199 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3b4ee-3b7a-4c76-a319-191b80de6d1f admin admin] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:24.373923 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3b4ee-3b7a-4c76-a319-191b80de6d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:24.375629 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3b4ee-3b7a-4c76-a319-191b80de6d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:24.377176 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad3b4ee-3b7a-4c76-a319-191b80de6d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:24.380859 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad3b4ee-3b7a-4c76-a319-191b80de6d1f admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:24.405982 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1427/2858] 10.222.0.57 () {58 vars in 1121 bytes} [Mon Jul 18 08:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 08:55:24.408030 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed12a0-3978-4384-b5fc-ecab3e0add2d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:24.428866 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed12a0-3978-4384-b5fc-ecab3e0add2d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:24.436680 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a5785-aa87-47cb-9bfc-d6f50fd3b9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:24.438442 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a5785-aa87-47cb-9bfc-d6f50fd3b9d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:24.438731 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a5785-aa87-47cb-9bfc-d6f50fd3b9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:24.439252 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a5785-aa87-47cb-9bfc-d6f50fd3b9d8 admin admin] PATH_INFO: `/v3/projects/9945807f747246729410568cee6fc929/users/343bb018170a49e09b72de95a19e769f/roles/587af95739954600bbf2411974601f74` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:24.464217 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a5785-aa87-47cb-9bfc-d6f50fd3b9d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9945807f747246729410568cee6fc929, user_id=343bb018170a49e09b72de95a19e769f, role_id=587af95739954600bbf2411974601f74)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:24.470416 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a5785-aa87-47cb-9bfc-d6f50fd3b9d8 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:24.650238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1432/2859] 10.222.0.57 () {60 vars in 1482 bytes} [Mon Jul 18 08:55:24 2022] PUT /identity/v3/projects/9945807f747246729410568cee6fc929/users/343bb018170a49e09b72de95a19e769f/roles/587af95739954600bbf2411974601f74 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:55:24.660010 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab5dff-6201-43bb-a800-847e37d10de5 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:24.661184 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab5dff-6201-43bb-a800-847e37d10de5 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:24.661989 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab5dff-6201-43bb-a800-847e37d10de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:24.699697 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d0ab5dff-6201-43bb-a800-847e37d10de5 None None] MFA Rules not processed for user `343bb018170a49e09b72de95a19e769f`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:24.701100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d0ab5dff-6201-43bb-a800-847e37d10de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:24.713216 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:24.854532 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1428/2860] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:24.868389 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a5785-aa87-47cb-9bfc-d6f50fd3b9d8 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:24.870127 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-e50a5785-aa87-47cb-9bfc-d6f50fd3b9d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:24.912992 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:24.964026 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec81f6-30dd-4f3d-9376-ed86a8b9c49a None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:24.964502 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec81f6-30dd-4f3d-9376-ed86a8b9c49a None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:24.965026 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec81f6-30dd-4f3d-9376-ed86a8b9c49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:24.969016 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a5785-aa87-47cb-9bfc-d6f50fd3b9d8 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:24.973580 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9bc0c-c23b-4060-bd00-48408624a3ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:24.974354 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9bc0c-c23b-4060-bd00-48408624a3ab service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:24.974845 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9bc0c-c23b-4060-bd00-48408624a3ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:24.975289 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9bc0c-c23b-4060-bd00-48408624a3ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:24.992224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9bc0c-c23b-4060-bd00-48408624a3ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:24.992224 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9bc0c-c23b-4060-bd00-48408624a3ab service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:25.001121 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-a2ec81f6-30dd-4f3d-9376-ed86a8b9c49a None None] MFA Rules not processed for user `ba4946e9565f4a608dd04fbc4de18dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:25.003340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-a2ec81f6-30dd-4f3d-9376-ed86a8b9c49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:25.010543 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1433/2861] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:55:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:55:25.098647 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1429/2862] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:26.978720 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310807d-58fb-46a2-bbeb-67f0a1ac0b31 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:26.979258 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310807d-58fb-46a2-bbeb-67f0a1ac0b31 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:26.979375 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310807d-58fb-46a2-bbeb-67f0a1ac0b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:27.010223 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-2310807d-58fb-46a2-bbeb-67f0a1ac0b31 None None] MFA Rules not processed for user `343bb018170a49e09b72de95a19e769f`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:27.011877 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2310807d-58fb-46a2-bbeb-67f0a1ac0b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:27.026895 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1434/2863] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:32.862828 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3b300-5e6d-4e6f-ba20-027bee34bb74 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:32.863812 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3b300-5e6d-4e6f-ba20-027bee34bb74 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:32.864272 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3b300-5e6d-4e6f-ba20-027bee34bb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:32.922888 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-36b3b300-5e6d-4e6f-ba20-027bee34bb74 None None] MFA Rules not processed for user `4449dce2c8b14df48b8746548e5ed55a`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:32.922888 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-36b3b300-5e6d-4e6f-ba20-027bee34bb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:33.025467 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1430/2864] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:32 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:33.042392 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310807d-58fb-46a2-bbeb-67f0a1ac0b31 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:33.044049 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2310807d-58fb-46a2-bbeb-67f0a1ac0b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:33.139723 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310807d-58fb-46a2-bbeb-67f0a1ac0b31 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:33.145363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c2cfc-b271-4203-a5e6-c5ca6580c448 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:33.146069 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c2cfc-b271-4203-a5e6-c5ca6580c448 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:33.146354 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c2cfc-b271-4203-a5e6-c5ca6580c448 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:33.146632 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c2cfc-b271-4203-a5e6-c5ca6580c448 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:33.148665 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-005c2cfc-b271-4203-a5e6-c5ca6580c448 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:33.239630 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c2cfc-b271-4203-a5e6-c5ca6580c448 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:33.241449 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c2cfc-b271-4203-a5e6-c5ca6580c448 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:33.255119 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1435/2865] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:55:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:55:38.509674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c3bf2-b35a-4093-b7b2-3216b700aaea None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:38.514438 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c3bf2-b35a-4093-b7b2-3216b700aaea None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:38.514438 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c3bf2-b35a-4093-b7b2-3216b700aaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:38.538028 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-d31c3bf2-b35a-4093-b7b2-3216b700aaea None None] MFA Rules not processed for user `124957514a4e4c35bed50a19de1dda6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:38.540225 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-d31c3bf2-b35a-4093-b7b2-3216b700aaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:38.553309 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1431/2866] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:43.247433 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=75 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:48.529393 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=84 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:49.740316 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c2cfc-b271-4203-a5e6-c5ca6580c448 service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:49.742644 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-005c2cfc-b271-4203-a5e6-c5ca6580c448 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:49.820531 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c2cfc-b271-4203-a5e6-c5ca6580c448 service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:49.824581 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3de93c-c825-47e9-8ce6-93d9d9a245e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:49.825245 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3de93c-c825-47e9-8ce6-93d9d9a245e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:49.826056 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3de93c-c825-47e9-8ce6-93d9d9a245e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:49.826369 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3de93c-c825-47e9-8ce6-93d9d9a245e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:49.828493 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ac3de93c-c825-47e9-8ce6-93d9d9a245e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:49.908756 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3de93c-c825-47e9-8ce6-93d9d9a245e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:49.910137 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3de93c-c825-47e9-8ce6-93d9d9a245e9 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:49.921197 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1436/2867] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:55:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:55:51.790801 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a7dfd-df1a-4697-978c-92e2ea0a4535 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:51.791401 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a7dfd-df1a-4697-978c-92e2ea0a4535 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:51.791640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a7dfd-df1a-4697-978c-92e2ea0a4535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:51.812781 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-284a7dfd-df1a-4697-978c-92e2ea0a4535 None None] MFA Rules not processed for user `2b03f8b8c51e472bbd4cf0afc9c6b632`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:51.814019 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-284a7dfd-df1a-4697-978c-92e2ea0a4535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:51.871233 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1432/2868] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Jul 18 08:55:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:51.885481 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3de93c-c825-47e9-8ce6-93d9d9a245e9 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:51.886950 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-ac3de93c-c825-47e9-8ce6-93d9d9a245e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:51.987373 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3de93c-c825-47e9-8ce6-93d9d9a245e9 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:52.000118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4c95e-3550-4d7f-9d09-92a6bc2e6457 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09a5c2e804504038817cad76544dce5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:52.001084 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4c95e-3550-4d7f-9d09-92a6bc2e6457 service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:52.001824 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4c95e-3550-4d7f-9d09-92a6bc2e6457 service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:52.002498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4c95e-3550-4d7f-9d09-92a6bc2e6457 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:52.017845 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4c95e-3550-4d7f-9d09-92a6bc2e6457 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:52.019998 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4c95e-3550-4d7f-9d09-92a6bc2e6457 service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:52.031732 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1437/2869] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Jul 18 08:55:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 08:55:53.078460 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725bbac-cb45-4484-8e0b-b7ca5dbf0429 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:53.079026 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725bbac-cb45-4484-8e0b-b7ca5dbf0429 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:53.079388 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725bbac-cb45-4484-8e0b-b7ca5dbf0429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:53.106733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-5725bbac-cb45-4484-8e0b-b7ca5dbf0429 None None] MFA Rules not processed for user `2b03f8b8c51e472bbd4cf0afc9c6b632`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:53.108552 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-5725bbac-cb45-4484-8e0b-b7ca5dbf0429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:53.116032 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1433/2870] 10.222.0.57 () {58 vars in 967 bytes} [Mon Jul 18 08:55:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:53.130185 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4c95e-3550-4d7f-9d09-92a6bc2e6457 service swift] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:53.142829 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4c95e-3550-4d7f-9d09-92a6bc2e6457 service swift] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:53.148410 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593732d-00bc-4eac-8060-5742633b2b7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09a5c2e804504038817cad76544dce5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:53.149094 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593732d-00bc-4eac-8060-5742633b2b7e service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:53.149374 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593732d-00bc-4eac-8060-5742633b2b7e service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:53.149647 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593732d-00bc-4eac-8060-5742633b2b7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:53.162033 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593732d-00bc-4eac-8060-5742633b2b7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:53.163638 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593732d-00bc-4eac-8060-5742633b2b7e service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:53.173586 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1438/2871] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Jul 18 08:55:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 08:55:53.920495 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ffb38-83dc-4ba8-aab3-bdb153fef323 None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:53.920773 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ffb38-83dc-4ba8-aab3-bdb153fef323 None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:53.921053 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ffb38-83dc-4ba8-aab3-bdb153fef323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:53.940936 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-709ffb38-83dc-4ba8-aab3-bdb153fef323 None None] MFA Rules not processed for user `2b03f8b8c51e472bbd4cf0afc9c6b632`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:53.942161 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-709ffb38-83dc-4ba8-aab3-bdb153fef323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:53.949799 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1434/2872] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Jul 18 08:55:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:53.961766 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593732d-00bc-4eac-8060-5742633b2b7e service swift] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:53.973507 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593732d-00bc-4eac-8060-5742633b2b7e service swift] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:53.978738 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65b3d9-f33f-4ce6-b400-773c6d792654 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09a5c2e804504038817cad76544dce5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:53.979387 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65b3d9-f33f-4ce6-b400-773c6d792654 service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:53.979687 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65b3d9-f33f-4ce6-b400-773c6d792654 service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:53.979976 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65b3d9-f33f-4ce6-b400-773c6d792654 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:53.990173 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65b3d9-f33f-4ce6-b400-773c6d792654 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:53.991817 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65b3d9-f33f-4ce6-b400-773c6d792654 service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:54.001498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1439/2873] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Jul 18 08:55:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 08:55:54.770733 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42393478-35bd-4532-a433-506e9a983a6b None None] REQUEST_METHOD: `POST` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:54.771245 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42393478-35bd-4532-a433-506e9a983a6b None None] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:54.771364 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42393478-35bd-4532-a433-506e9a983a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:54.791535 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.auth.core [None req-42393478-35bd-4532-a433-506e9a983a6b None None] MFA Rules not processed for user `2b03f8b8c51e472bbd4cf0afc9c6b632`. Rule list: `[]` (Enabled: `True`). {{(pid=80338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:54.792794 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-42393478-35bd-4532-a433-506e9a983a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:54.800413 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1435/2874] 10.222.0.57 () {58 vars in 967 bytes} [Mon Jul 18 08:55:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:55:54.826348 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65b3d9-f33f-4ce6-b400-773c6d792654 service swift] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:54.836354 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65b3d9-f33f-4ce6-b400-773c6d792654 service swift] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:54.841592 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9f6e8-f0b2-47a0-b0c9-14fcc891f0d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09a5c2e804504038817cad76544dce5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:54.842412 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9f6e8-f0b2-47a0-b0c9-14fcc891f0d3 service swift] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:54.842721 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9f6e8-f0b2-47a0-b0c9-14fcc891f0d3 service swift] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:54.842997 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9f6e8-f0b2-47a0-b0c9-14fcc891f0d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:54.852121 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9f6e8-f0b2-47a0-b0c9-14fcc891f0d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:54.853725 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9f6e8-f0b2-47a0-b0c9-14fcc891f0d3 service swift] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:54.859375 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=100 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:54.863384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1440/2875] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Jul 18 08:55:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 08:55:57.649104 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42393478-35bd-4532-a433-506e9a983a6b None None] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:57.718123 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42393478-35bd-4532-a433-506e9a983a6b None None] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:57.722820 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4e75b-ee98-48f9-a607-9f30cd944b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:57.723717 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4e75b-ee98-48f9-a607-9f30cd944b8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:57.724007 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4e75b-ee98-48f9-a607-9f30cd944b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:57.724286 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4e75b-ee98-48f9-a607-9f30cd944b8a admin admin] PATH_INFO: `/v3/users/343bb018170a49e09b72de95a19e769f` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:57.725517 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4e75b-ee98-48f9-a607-9f30cd944b8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=343bb018170a49e09b72de95a19e769f)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:57.727447 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4e75b-ee98-48f9-a607-9f30cd944b8a admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:57.999868 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-eec4e75b-ee98-48f9-a607-9f30cd944b8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '343bb018170a49e09b72de95a19e769f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:58.000546 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-eec4e75b-ee98-48f9-a607-9f30cd944b8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '343bb018170a49e09b72de95a19e769f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:58.001146 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-eec4e75b-ee98-48f9-a607-9f30cd944b8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '343bb018170a49e09b72de95a19e769f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:58.013663 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-eec4e75b-ee98-48f9-a607-9f30cd944b8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '343bb018170a49e09b72de95a19e769f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:58.021217 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-eec4e75b-ee98-48f9-a607-9f30cd944b8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '343bb018170a49e09b72de95a19e769f'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:58.101273 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1436/2876] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:55:57 2022] DELETE /identity/v3/users/343bb018170a49e09b72de95a19e769f => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:55:58.106036 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5b64a-7f0c-4c86-9fed-d0bed6a00e08 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:58.107010 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5b64a-7f0c-4c86-9fed-d0bed6a00e08 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:58.107118 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5b64a-7f0c-4c86-9fed-d0bed6a00e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:58.126609 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-69d5b64a-7f0c-4c86-9fed-d0bed6a00e08 None None] MFA Rules not processed for user `93ad24fe28ec40dbbca364cbdb34972d`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:55:58.127985 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-69d5b64a-7f0c-4c86-9fed-d0bed6a00e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:58.190000 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1441/2877] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 08:55:58.201214 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4e75b-ee98-48f9-a607-9f30cd944b8a admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:58.203045 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-eec4e75b-ee98-48f9-a607-9f30cd944b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:58.212256 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:55:58.274387 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4e75b-ee98-48f9-a607-9f30cd944b8a admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:58.278119 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6c1f3-a5ac-41fc-8766-5c3a2ca37c00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:58.279353 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6c1f3-a5ac-41fc-8766-5c3a2ca37c00 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:58.279641 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6c1f3-a5ac-41fc-8766-5c3a2ca37c00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:58.279946 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6c1f3-a5ac-41fc-8766-5c3a2ca37c00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:58.289665 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6c1f3-a5ac-41fc-8766-5c3a2ca37c00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:58.291106 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6c1f3-a5ac-41fc-8766-5c3a2ca37c00 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:58.302853 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1437/2878] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:55:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:55:58.594711 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5b64a-7f0c-4c86-9fed-d0bed6a00e08 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:55:58.596273 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-69d5b64a-7f0c-4c86-9fed-d0bed6a00e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:55:58.618863 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5b64a-7f0c-4c86-9fed-d0bed6a00e08 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:55:58.629216 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7d068-0fcb-4b4b-9947-fd30c6f5536d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:55:58.629216 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7d068-0fcb-4b4b-9947-fd30c6f5536d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:55:58.629216 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7d068-0fcb-4b4b-9947-fd30c6f5536d admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:55:58.629216 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7d068-0fcb-4b4b-9947-fd30c6f5536d admin admin] PATH_INFO: `/v3/projects/9945807f747246729410568cee6fc929` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:55:58.630158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7d068-0fcb-4b4b-9947-fd30c6f5536d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9945807f747246729410568cee6fc929)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:55:58.630158 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7d068-0fcb-4b4b-9947-fd30c6f5536d admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:55:58.793343 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-94a7d068-0fcb-4b4b-9947-fd30c6f5536d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9945807f747246729410568cee6fc929', 'request_id': 'req-94a7d068-0fcb-4b4b-9947-fd30c6f5536d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:58.799476 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-94a7d068-0fcb-4b4b-9947-fd30c6f5536d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9945807f747246729410568cee6fc929', 'request_id': 'req-94a7d068-0fcb-4b4b-9947-fd30c6f5536d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:58.805958 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-94a7d068-0fcb-4b4b-9947-fd30c6f5536d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9945807f747246729410568cee6fc929', 'request_id': 'req-94a7d068-0fcb-4b4b-9947-fd30c6f5536d'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:58.885108 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-94a7d068-0fcb-4b4b-9947-fd30c6f5536d admin admin] The token cache is being invalidate because project 9945807f747246729410568cee6fc929 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:55:58.885550 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-94a7d068-0fcb-4b4b-9947-fd30c6f5536d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:58.886614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-94a7d068-0fcb-4b4b-9947-fd30c6f5536d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:55:58.890626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1442/2879] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:55:58 2022] DELETE /identity/v3/projects/9945807f747246729410568cee6fc929 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:56:08.453118 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=24 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:56:08.878423 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:56:24.067156 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6c1f3-a5ac-41fc-8766-5c3a2ca37c00 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:56:24.068704 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e2f6c1f3-a5ac-41fc-8766-5c3a2ca37c00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:56:24.081164 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6c1f3-a5ac-41fc-8766-5c3a2ca37c00 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:56:24.132712 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5d16b-f5ad-4b71-9a59-e398066b80e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:56:24.133293 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5d16b-f5ad-4b71-9a59-e398066b80e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:56:24.133572 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5d16b-f5ad-4b71-9a59-e398066b80e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:56:24.133846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5d16b-f5ad-4b71-9a59-e398066b80e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:56:24.135633 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e4c5d16b-f5ad-4b71-9a59-e398066b80e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:56:24.212723 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c5d16b-f5ad-4b71-9a59-e398066b80e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:56:24.214170 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c5d16b-f5ad-4b71-9a59-e398066b80e5 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:56:24.226215 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1438/2880] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:56:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:56:24.813003 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7d068-0fcb-4b4b-9947-fd30c6f5536d admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:56:24.814440 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-94a7d068-0fcb-4b4b-9947-fd30c6f5536d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:56:24.876417 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7d068-0fcb-4b4b-9947-fd30c6f5536d admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:56:24.880040 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef11ae6-c4a3-4d40-838a-155f22cacbff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5f9a129021445ae8154a5e6941291e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:56:24.880680 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef11ae6-c4a3-4d40-838a-155f22cacbff service cinder] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:56:24.880965 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef11ae6-c4a3-4d40-838a-155f22cacbff service cinder] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:56:24.881241 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef11ae6-c4a3-4d40-838a-155f22cacbff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:56:24.882999 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-cef11ae6-c4a3-4d40-838a-155f22cacbff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:56:24.897088 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef11ae6-c4a3-4d40-838a-155f22cacbff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:56:24.898512 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef11ae6-c4a3-4d40-838a-155f22cacbff service cinder] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:56:24.909331 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1443/2881] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:56:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:56:25.701815 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5d16b-f5ad-4b71-9a59-e398066b80e5 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:56:25.709211 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-e4c5d16b-f5ad-4b71-9a59-e398066b80e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:56:25.714358 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5d16b-f5ad-4b71-9a59-e398066b80e5 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:56:25.718454 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1992c3-95c8-4c30-8775-b4393c157f08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:56:25.719090 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1992c3-95c8-4c30-8775-b4393c157f08 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:56:25.719369 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1992c3-95c8-4c30-8775-b4393c157f08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:56:25.719642 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1992c3-95c8-4c30-8775-b4393c157f08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:56:25.721577 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3a1992c3-95c8-4c30-8775-b4393c157f08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:56:25.785446 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1992c3-95c8-4c30-8775-b4393c157f08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:56:25.786867 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1992c3-95c8-4c30-8775-b4393c157f08 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:56:25.797208 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1439/2882] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:56:25 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 08:56:34.904541 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=26 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:56:35.792072 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=84 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:56:40.793795 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4303d5-1da6-4511-93d2-77fed7875bb7 None None] REQUEST_METHOD: `POST` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:56:40.794614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4303d5-1da6-4511-93d2-77fed7875bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:56:40.794614 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4303d5-1da6-4511-93d2-77fed7875bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:56:40.816651 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.auth.core [None req-0a4303d5-1da6-4511-93d2-77fed7875bb7 None None] MFA Rules not processed for user `b9f08551f4ff4bbd9c48c438a518c8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80339) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 08:56:40.817782 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0a4303d5-1da6-4511-93d2-77fed7875bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:56:40.875597 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1444/2883] 10.222.0.57 () {58 vars in 957 bytes} [Mon Jul 18 08:56:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 08:56:41.537827 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1992c3-95c8-4c30-8775-b4393c157f08 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:56:41.539409 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3a1992c3-95c8-4c30-8775-b4393c157f08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:56:41.628351 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1992c3-95c8-4c30-8775-b4393c157f08 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:56:41.633333 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4362b4-5ff5-4bb1-92e4-b89756fbc36d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd606fd7ef269460a8110223932f02d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:56:41.633980 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4362b4-5ff5-4bb1-92e4-b89756fbc36d service placement] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:56:41.634290 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4362b4-5ff5-4bb1-92e4-b89756fbc36d service placement] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:56:41.634563 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4362b4-5ff5-4bb1-92e4-b89756fbc36d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:56:41.636362 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1d4362b4-5ff5-4bb1-92e4-b89756fbc36d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:56:41.648068 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4362b4-5ff5-4bb1-92e4-b89756fbc36d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:56:41.649640 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4362b4-5ff5-4bb1-92e4-b89756fbc36d service placement] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:56:41.660294 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1440/2884] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:56:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:56:44.395008 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4303d5-1da6-4511-93d2-77fed7875bb7 None None] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:56:44.396518 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0a4303d5-1da6-4511-93d2-77fed7875bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:56:44.408758 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4303d5-1da6-4511-93d2-77fed7875bb7 None None] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:56:44.412415 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828ddbd-e353-4579-9b7b-6043f796ca29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:56:44.413061 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828ddbd-e353-4579-9b7b-6043f796ca29 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:56:44.413348 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828ddbd-e353-4579-9b7b-6043f796ca29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:56:44.413626 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828ddbd-e353-4579-9b7b-6043f796ca29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:56:44.415401 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-8828ddbd-e353-4579-9b7b-6043f796ca29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:56:44.426110 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828ddbd-e353-4579-9b7b-6043f796ca29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:56:44.427562 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828ddbd-e353-4579-9b7b-6043f796ca29 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:56:44.436720 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1445/2885] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:56:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:56:45.288815 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4362b4-5ff5-4bb1-92e4-b89756fbc36d service placement] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:56:45.290347 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-1d4362b4-5ff5-4bb1-92e4-b89756fbc36d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:56:45.307753 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4362b4-5ff5-4bb1-92e4-b89756fbc36d service placement] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:56:45.313752 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91afad-a1eb-4bd2-8f4f-b27a7bcbfcea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:56:45.320745 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91afad-a1eb-4bd2-8f4f-b27a7bcbfcea service nova] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:56:45.320745 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91afad-a1eb-4bd2-8f4f-b27a7bcbfcea service nova] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:56:45.320745 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91afad-a1eb-4bd2-8f4f-b27a7bcbfcea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:56:45.320745 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-df91afad-a1eb-4bd2-8f4f-b27a7bcbfcea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:56:45.333094 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91afad-a1eb-4bd2-8f4f-b27a7bcbfcea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:56:45.334678 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91afad-a1eb-4bd2-8f4f-b27a7bcbfcea service nova] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:56:45.345921 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1441/2886] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:56:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:56:54.432567 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=31 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:56:55.341021 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=38 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:57:37.016581 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828ddbd-e353-4579-9b7b-6043f796ca29 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:57:37.030432 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828ddbd-e353-4579-9b7b-6043f796ca29 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:57:37.036860 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95983c-3292-49b7-9d89-1882c0a757eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:57:37.039560 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95983c-3292-49b7-9d89-1882c0a757eb service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:57:37.039560 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95983c-3292-49b7-9d89-1882c0a757eb service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:57:37.039560 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95983c-3292-49b7-9d89-1882c0a757eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:57:37.039928 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bb95983c-3292-49b7-9d89-1882c0a757eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:57:37.052405 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95983c-3292-49b7-9d89-1882c0a757eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:57:37.053996 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95983c-3292-49b7-9d89-1882c0a757eb service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:57:37.064661 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1446/2887] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:57:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:57:47.060116 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=6 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:57:47.538686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91afad-a1eb-4bd2-8f4f-b27a7bcbfcea service nova] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:57:47.551686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91afad-a1eb-4bd2-8f4f-b27a7bcbfcea service nova] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:57:47.558029 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290686e-cb25-4d51-bd5f-1e6815ddf72a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd606fd7ef269460a8110223932f02d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:57:47.558916 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290686e-cb25-4d51-bd5f-1e6815ddf72a service placement] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:57:47.559224 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290686e-cb25-4d51-bd5f-1e6815ddf72a service placement] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:57:47.559521 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290686e-cb25-4d51-bd5f-1e6815ddf72a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:57:47.561429 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-6290686e-cb25-4d51-bd5f-1e6815ddf72a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:57:47.575212 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290686e-cb25-4d51-bd5f-1e6815ddf72a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:57:47.577000 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290686e-cb25-4d51-bd5f-1e6815ddf72a service placement] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:57:47.587887 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1442/2888] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:57:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:57:47.700403 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95983c-3292-49b7-9d89-1882c0a757eb service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:57:47.701536 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-bb95983c-3292-49b7-9d89-1882c0a757eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:57:47.772222 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95983c-3292-49b7-9d89-1882c0a757eb service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:57:47.777322 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479e8cf-4fb9-4590-97b2-21c49997dead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:57:47.778203 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479e8cf-4fb9-4590-97b2-21c49997dead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:57:47.778482 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479e8cf-4fb9-4590-97b2-21c49997dead admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:57:47.778749 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479e8cf-4fb9-4590-97b2-21c49997dead admin admin] PATH_INFO: `/v3/users/b9f08551f4ff4bbd9c48c438a518c8d0` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:57:47.780018 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479e8cf-4fb9-4590-97b2-21c49997dead admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9f08551f4ff4bbd9c48c438a518c8d0)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:57:47.781951 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479e8cf-4fb9-4590-97b2-21c49997dead admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:57:47.974331 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6479e8cf-4fb9-4590-97b2-21c49997dead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9f08551f4ff4bbd9c48c438a518c8d0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:47.974983 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6479e8cf-4fb9-4590-97b2-21c49997dead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9f08551f4ff4bbd9c48c438a518c8d0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:47.975622 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6479e8cf-4fb9-4590-97b2-21c49997dead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9f08551f4ff4bbd9c48c438a518c8d0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:47.988031 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6479e8cf-4fb9-4590-97b2-21c49997dead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9f08551f4ff4bbd9c48c438a518c8d0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:47.995989 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-6479e8cf-4fb9-4590-97b2-21c49997dead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9f08551f4ff4bbd9c48c438a518c8d0'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:48.068942 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1447/2889] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:57:47 2022] DELETE /identity/v3/users/b9f08551f4ff4bbd9c48c438a518c8d0 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:57:48.324384 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290686e-cb25-4d51-bd5f-1e6815ddf72a service placement] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:57:48.335518 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290686e-cb25-4d51-bd5f-1e6815ddf72a service placement] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:57:48.400372 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594bd80-0ac4-4ac2-b015-ffc9a599e759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:57:48.401302 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594bd80-0ac4-4ac2-b015-ffc9a599e759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:57:48.401548 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594bd80-0ac4-4ac2-b015-ffc9a599e759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:57:48.401860 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594bd80-0ac4-4ac2-b015-ffc9a599e759 admin admin] PATH_INFO: `/v3/projects/3df75aa755784fee814abfd669ce02da` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:57:48.403345 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594bd80-0ac4-4ac2-b015-ffc9a599e759 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3df75aa755784fee814abfd669ce02da)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:57:48.405744 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594bd80-0ac4-4ac2-b015-ffc9a599e759 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:57:48.552833 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3594bd80-0ac4-4ac2-b015-ffc9a599e759 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3df75aa755784fee814abfd669ce02da', 'request_id': 'req-3594bd80-0ac4-4ac2-b015-ffc9a599e759'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:48.566046 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3594bd80-0ac4-4ac2-b015-ffc9a599e759 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3df75aa755784fee814abfd669ce02da', 'request_id': 'req-3594bd80-0ac4-4ac2-b015-ffc9a599e759'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:48.575839 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479e8cf-4fb9-4590-97b2-21c49997dead admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:57:48.577403 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-6479e8cf-4fb9-4590-97b2-21c49997dead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:57:48.577728 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3594bd80-0ac4-4ac2-b015-ffc9a599e759 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3df75aa755784fee814abfd669ce02da', 'request_id': 'req-3594bd80-0ac4-4ac2-b015-ffc9a599e759'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:48.645363 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479e8cf-4fb9-4590-97b2-21c49997dead admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:57:48.672974 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3594bd80-0ac4-4ac2-b015-ffc9a599e759 admin admin] The token cache is being invalidate because project 3df75aa755784fee814abfd669ce02da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:57:48.673264 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3594bd80-0ac4-4ac2-b015-ffc9a599e759 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:48.674384 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-3594bd80-0ac4-4ac2-b015-ffc9a599e759 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:48.678071 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1443/2890] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:57:48 2022] DELETE /identity/v3/projects/3df75aa755784fee814abfd669ce02da => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:57:48.695150 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:57:48.718564 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f3d52-076a-49fa-b4a5-55312fe73550 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:57:48.719508 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f3d52-076a-49fa-b4a5-55312fe73550 service neutron] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:57:48.719926 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f3d52-076a-49fa-b4a5-55312fe73550 service neutron] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:57:48.720243 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f3d52-076a-49fa-b4a5-55312fe73550 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:57:48.722256 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-664f3d52-076a-49fa-b4a5-55312fe73550 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:57:48.798290 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f3d52-076a-49fa-b4a5-55312fe73550 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:57:48.799801 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f3d52-076a-49fa-b4a5-55312fe73550 service neutron] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:57:48.812865 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1448/2891] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:57:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:57:56.174398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594bd80-0ac4-4ac2-b015-ffc9a599e759 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:57:56.175965 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-3594bd80-0ac4-4ac2-b015-ffc9a599e759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:57:56.202097 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594bd80-0ac4-4ac2-b015-ffc9a599e759 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:57:56.206949 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb0c8f-eeab-41e7-9338-957fc5a62575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:57:56.207940 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb0c8f-eeab-41e7-9338-957fc5a62575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:57:56.208225 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb0c8f-eeab-41e7-9338-957fc5a62575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:57:56.208505 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb0c8f-eeab-41e7-9338-957fc5a62575 admin admin] PATH_INFO: `/v3/users/ba4946e9565f4a608dd04fbc4de18dad` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:57:56.209709 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb0c8f-eeab-41e7-9338-957fc5a62575 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba4946e9565f4a608dd04fbc4de18dad)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:57:56.211813 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb0c8f-eeab-41e7-9338-957fc5a62575 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:57:56.540686 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-cdbb0c8f-eeab-41e7-9338-957fc5a62575 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba4946e9565f4a608dd04fbc4de18dad'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:56.541378 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-cdbb0c8f-eeab-41e7-9338-957fc5a62575 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba4946e9565f4a608dd04fbc4de18dad'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:56.541971 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-cdbb0c8f-eeab-41e7-9338-957fc5a62575 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba4946e9565f4a608dd04fbc4de18dad'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:56.554100 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-cdbb0c8f-eeab-41e7-9338-957fc5a62575 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba4946e9565f4a608dd04fbc4de18dad'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:56.559804 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:57:56.562014 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-cdbb0c8f-eeab-41e7-9338-957fc5a62575 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba4946e9565f4a608dd04fbc4de18dad'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:56.641989 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1444/2892] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:57:56 2022] DELETE /identity/v3/users/ba4946e9565f4a608dd04fbc4de18dad => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:57:56.918030 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f3d52-076a-49fa-b4a5-55312fe73550 service neutron] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:57:56.932356 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f3d52-076a-49fa-b4a5-55312fe73550 service neutron] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:57:56.990436 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556b7c0-3b9f-43f1-acb0-39a14a444027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:57:56.991461 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556b7c0-3b9f-43f1-acb0-39a14a444027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:57:56.991760 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556b7c0-3b9f-43f1-acb0-39a14a444027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:57:56.992043 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556b7c0-3b9f-43f1-acb0-39a14a444027 admin admin] PATH_INFO: `/v3/projects/d37f9217ffb94786830f0f6c5970b762` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:57:56.993361 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9556b7c0-3b9f-43f1-acb0-39a14a444027 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d37f9217ffb94786830f0f6c5970b762)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:57:56.995298 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9556b7c0-3b9f-43f1-acb0-39a14a444027 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:57:57.128726 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9556b7c0-3b9f-43f1-acb0-39a14a444027 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd37f9217ffb94786830f0f6c5970b762', 'request_id': 'req-9556b7c0-3b9f-43f1-acb0-39a14a444027'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:57.135373 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9556b7c0-3b9f-43f1-acb0-39a14a444027 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd37f9217ffb94786830f0f6c5970b762', 'request_id': 'req-9556b7c0-3b9f-43f1-acb0-39a14a444027'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:57.142008 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9556b7c0-3b9f-43f1-acb0-39a14a444027 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd37f9217ffb94786830f0f6c5970b762', 'request_id': 'req-9556b7c0-3b9f-43f1-acb0-39a14a444027'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:57.223530 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9556b7c0-3b9f-43f1-acb0-39a14a444027 admin admin] The token cache is being invalidate because project d37f9217ffb94786830f0f6c5970b762 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:57:57.224076 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9556b7c0-3b9f-43f1-acb0-39a14a444027 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:57.224849 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-9556b7c0-3b9f-43f1-acb0-39a14a444027 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:57.226515 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1449/2893] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:57:56 2022] DELETE /identity/v3/projects/d37f9217ffb94786830f0f6c5970b762 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:57:57.230851 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb0c8f-eeab-41e7-9338-957fc5a62575 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:57:57.232392 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-cdbb0c8f-eeab-41e7-9338-957fc5a62575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:57:57.253199 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb0c8f-eeab-41e7-9338-957fc5a62575 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:57:57.320803 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5493bbb-f4a5-4121-b698-bbc7bcf3ca47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:57:57.322631 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5493bbb-f4a5-4121-b698-bbc7bcf3ca47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:57:57.323120 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5493bbb-f4a5-4121-b698-bbc7bcf3ca47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:57:57.323463 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5493bbb-f4a5-4121-b698-bbc7bcf3ca47 admin admin] PATH_INFO: `/v3/users/bb00f4132d9247309511245686efbd43` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:57:57.324834 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5493bbb-f4a5-4121-b698-bbc7bcf3ca47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb00f4132d9247309511245686efbd43)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:57:57.326941 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5493bbb-f4a5-4121-b698-bbc7bcf3ca47 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:57:57.542581 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f5493bbb-f4a5-4121-b698-bbc7bcf3ca47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb00f4132d9247309511245686efbd43'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:57.543358 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f5493bbb-f4a5-4121-b698-bbc7bcf3ca47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb00f4132d9247309511245686efbd43'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:57.544005 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f5493bbb-f4a5-4121-b698-bbc7bcf3ca47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb00f4132d9247309511245686efbd43'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:57.558391 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f5493bbb-f4a5-4121-b698-bbc7bcf3ca47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb00f4132d9247309511245686efbd43'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:57.566751 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-f5493bbb-f4a5-4121-b698-bbc7bcf3ca47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb00f4132d9247309511245686efbd43'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:57.693340 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1445/2894] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 08:57:57 2022] DELETE /identity/v3/users/bb00f4132d9247309511245686efbd43 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:57:58.087204 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556b7c0-3b9f-43f1-acb0-39a14a444027 admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:57:58.089816 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-9556b7c0-3b9f-43f1-acb0-39a14a444027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:57:58.106491 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556b7c0-3b9f-43f1-acb0-39a14a444027 admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:57:58.128824 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:57:58.213606 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a8d89-a17d-48ed-821b-f0a127080a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:57:58.214498 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a8d89-a17d-48ed-821b-f0a127080a0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:57:58.214785 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a8d89-a17d-48ed-821b-f0a127080a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:57:58.215062 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a8d89-a17d-48ed-821b-f0a127080a0a admin admin] PATH_INFO: `/v3/projects/486e45a580e64f3d8b3f063a1d19fcd7` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:57:58.216958 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a8d89-a17d-48ed-821b-f0a127080a0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=486e45a580e64f3d8b3f063a1d19fcd7)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:57:58.219031 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a8d89-a17d-48ed-821b-f0a127080a0a admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:57:58.391805 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a64a8d89-a17d-48ed-821b-f0a127080a0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '486e45a580e64f3d8b3f063a1d19fcd7', 'request_id': 'req-a64a8d89-a17d-48ed-821b-f0a127080a0a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:58.397370 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a64a8d89-a17d-48ed-821b-f0a127080a0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '486e45a580e64f3d8b3f063a1d19fcd7', 'request_id': 'req-a64a8d89-a17d-48ed-821b-f0a127080a0a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:58.405269 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a64a8d89-a17d-48ed-821b-f0a127080a0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '486e45a580e64f3d8b3f063a1d19fcd7', 'request_id': 'req-a64a8d89-a17d-48ed-821b-f0a127080a0a'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:58.492444 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a64a8d89-a17d-48ed-821b-f0a127080a0a admin admin] The token cache is being invalidate because project 486e45a580e64f3d8b3f063a1d19fcd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 08:57:58.492766 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a64a8d89-a17d-48ed-821b-f0a127080a0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:58.493519 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-a64a8d89-a17d-48ed-821b-f0a127080a0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 08:57:58.495972 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1450/2895] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 08:57:58 2022] DELETE /identity/v3/projects/486e45a580e64f3d8b3f063a1d19fcd7 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 08:58:07.689003 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=56,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:58:08.488833 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:58:42.747811 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5493bbb-f4a5-4121-b698-bbc7bcf3ca47 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:58:42.749343 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f5493bbb-f4a5-4121-b698-bbc7bcf3ca47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:58:42.809851 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5493bbb-f4a5-4121-b698-bbc7bcf3ca47 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:58:42.862356 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab389fe-baa9-4b6c-8298-9c1d62258b4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:58:42.863245 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab389fe-baa9-4b6c-8298-9c1d62258b4a service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:58:42.863637 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab389fe-baa9-4b6c-8298-9c1d62258b4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:58:42.863946 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab389fe-baa9-4b6c-8298-9c1d62258b4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:58:42.865782 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-eab389fe-baa9-4b6c-8298-9c1d62258b4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:58:42.875357 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab389fe-baa9-4b6c-8298-9c1d62258b4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:58:42.876820 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab389fe-baa9-4b6c-8298-9c1d62258b4a service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:58:42.885604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1446/2896] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:58:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:58:42.982342 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a8d89-a17d-48ed-821b-f0a127080a0a admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:58:42.983931 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-a64a8d89-a17d-48ed-821b-f0a127080a0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:58:43.057815 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a8d89-a17d-48ed-821b-f0a127080a0a admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:58:43.065277 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04cac7-c2e7-478b-9fbc-ffe23987b326 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84cba474fc7c470ea119de697b9d7ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:58:43.065277 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04cac7-c2e7-478b-9fbc-ffe23987b326 service nova] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:58:43.065277 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04cac7-c2e7-478b-9fbc-ffe23987b326 service nova] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:58:43.065277 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04cac7-c2e7-478b-9fbc-ffe23987b326 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:58:43.066855 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2d04cac7-c2e7-478b-9fbc-ffe23987b326 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:58:43.080249 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04cac7-c2e7-478b-9fbc-ffe23987b326 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:58:43.082445 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04cac7-c2e7-478b-9fbc-ffe23987b326 service nova] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:58:43.093900 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1451/2897] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:58:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:58:43.967542 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab389fe-baa9-4b6c-8298-9c1d62258b4a service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:58:43.968974 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-eab389fe-baa9-4b6c-8298-9c1d62258b4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:58:43.979972 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab389fe-baa9-4b6c-8298-9c1d62258b4a service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:58:43.983566 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca71f9-e014-4df0-89cd-ec5c438374c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:58:43.984251 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca71f9-e014-4df0-89cd-ec5c438374c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:58:43.984516 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca71f9-e014-4df0-89cd-ec5c438374c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:58:43.984792 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca71f9-e014-4df0-89cd-ec5c438374c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:58:43.986519 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-33ca71f9-e014-4df0-89cd-ec5c438374c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:58:44.000478 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca71f9-e014-4df0-89cd-ec5c438374c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:58:44.001894 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca71f9-e014-4df0-89cd-ec5c438374c0 service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:58:44.013200 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1447/2898] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:58:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:58:53.089351 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=34 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:58:54.008430 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=52 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 08:59:15.743832 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04cac7-c2e7-478b-9fbc-ffe23987b326 service nova] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 08:59:15.745238 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-2d04cac7-c2e7-478b-9fbc-ffe23987b326 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:59:15.814218 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04cac7-c2e7-478b-9fbc-ffe23987b326 service nova] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 08:59:15.819383 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe31f35-ce92-4d93-b4c9-62640e8e132e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd606fd7ef269460a8110223932f02d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 08:59:15.820731 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe31f35-ce92-4d93-b4c9-62640e8e132e service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 08:59:15.821241 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe31f35-ce92-4d93-b4c9-62640e8e132e service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 08:59:15.821532 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe31f35-ce92-4d93-b4c9-62640e8e132e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 08:59:15.823306 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0fe31f35-ce92-4d93-b4c9-62640e8e132e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 08:59:15.834700 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe31f35-ce92-4d93-b4c9-62640e8e132e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 08:59:15.836780 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe31f35-ce92-4d93-b4c9-62640e8e132e service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 08:59:15.847179 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1452/2899] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 08:59:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 08:59:25.842664 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=32 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 09:00:21.507865 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca71f9-e014-4df0-89cd-ec5c438374c0 service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 09:00:21.509297 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-33ca71f9-e014-4df0-89cd-ec5c438374c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 09:00:21.520998 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca71f9-e014-4df0-89cd-ec5c438374c0 service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 09:00:21.524737 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52709270-f4eb-47fa-9ef2-92bd59ee98ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24a6b794f5774e35b67e9ec4358ffc66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 09:00:21.525367 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52709270-f4eb-47fa-9ef2-92bd59ee98ef service neutron] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 09:00:21.525650 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52709270-f4eb-47fa-9ef2-92bd59ee98ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 09:00:21.525933 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52709270-f4eb-47fa-9ef2-92bd59ee98ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 09:00:21.527777 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-52709270-f4eb-47fa-9ef2-92bd59ee98ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 09:00:21.594126 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52709270-f4eb-47fa-9ef2-92bd59ee98ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 09:00:21.595568 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52709270-f4eb-47fa-9ef2-92bd59ee98ef service neutron] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 09:00:21.605649 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1448/2900] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 09:00:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 09:00:25.116739 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe31f35-ce92-4d93-b4c9-62640e8e132e service placement] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 09:00:25.118213 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-0fe31f35-ce92-4d93-b4c9-62640e8e132e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 09:00:25.131954 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe31f35-ce92-4d93-b4c9-62640e8e132e service placement] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 09:00:25.138490 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5cbc5-03be-4b8c-b299-35b9e19765cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd606fd7ef269460a8110223932f02d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 09:00:25.139154 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5cbc5-03be-4b8c-b299-35b9e19765cd service placement] REQUEST_METHOD: `GET` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 09:00:25.139441 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5cbc5-03be-4b8c-b299-35b9e19765cd service placement] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 09:00:25.139747 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5cbc5-03be-4b8c-b299-35b9e19765cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 09:00:25.141536 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-37f5cbc5-03be-4b8c-b299-35b9e19765cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 09:00:25.154458 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5cbc5-03be-4b8c-b299-35b9e19765cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 09:00:25.156237 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5cbc5-03be-4b8c-b299-35b9e19765cd service placement] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 09:00:25.172362 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1453/2901] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 09:00:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 09:00:28.912445 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52709270-f4eb-47fa-9ef2-92bd59ee98ef service neutron] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 09:00:28.914074 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-52709270-f4eb-47fa-9ef2-92bd59ee98ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 09:00:28.926604 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52709270-f4eb-47fa-9ef2-92bd59ee98ef service neutron] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 09:00:28.931237 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542042b-1e42-4cfa-b929-d26e074e5a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 09:00:28.932174 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542042b-1e42-4cfa-b929-d26e074e5a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 09:00:28.932459 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542042b-1e42-4cfa-b929-d26e074e5a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 09:00:28.932736 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542042b-1e42-4cfa-b929-d26e074e5a2b admin admin] PATH_INFO: `/v3/users/124957514a4e4c35bed50a19de1dda6d` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 09:00:28.934523 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542042b-1e42-4cfa-b929-d26e074e5a2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=124957514a4e4c35bed50a19de1dda6d)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 09:00:28.936475 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542042b-1e42-4cfa-b929-d26e074e5a2b admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 09:00:29.142427 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b542042b-1e42-4cfa-b929-d26e074e5a2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '124957514a4e4c35bed50a19de1dda6d'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 09:00:29.143438 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b542042b-1e42-4cfa-b929-d26e074e5a2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '124957514a4e4c35bed50a19de1dda6d'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 09:00:29.144088 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b542042b-1e42-4cfa-b929-d26e074e5a2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '124957514a4e4c35bed50a19de1dda6d'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 09:00:29.156711 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b542042b-1e42-4cfa-b929-d26e074e5a2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '124957514a4e4c35bed50a19de1dda6d'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 09:00:29.163557 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-b542042b-1e42-4cfa-b929-d26e074e5a2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '124957514a4e4c35bed50a19de1dda6d'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 09:00:29.242848 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1449/2902] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 09:00:28 2022] DELETE /identity/v3/users/124957514a4e4c35bed50a19de1dda6d => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 09:00:29.501660 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5cbc5-03be-4b8c-b299-35b9e19765cd service placement] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 09:00:29.512871 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5cbc5-03be-4b8c-b299-35b9e19765cd service placement] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 09:00:29.572122 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3b00e-f887-484d-a8bb-f443bf09db1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 09:00:29.572932 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3b00e-f887-484d-a8bb-f443bf09db1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 09:00:29.573208 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3b00e-f887-484d-a8bb-f443bf09db1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 09:00:29.573479 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3b00e-f887-484d-a8bb-f443bf09db1c admin admin] PATH_INFO: `/v3/projects/98f69056ca3e4e7c83f6c36c8e16c16c` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 09:00:29.574788 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3b00e-f887-484d-a8bb-f443bf09db1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=98f69056ca3e4e7c83f6c36c8e16c16c)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 09:00:29.576766 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3b00e-f887-484d-a8bb-f443bf09db1c admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 09:00:29.706902 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-13a3b00e-f887-484d-a8bb-f443bf09db1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98f69056ca3e4e7c83f6c36c8e16c16c', 'request_id': 'req-13a3b00e-f887-484d-a8bb-f443bf09db1c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 09:00:29.712426 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-13a3b00e-f887-484d-a8bb-f443bf09db1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98f69056ca3e4e7c83f6c36c8e16c16c', 'request_id': 'req-13a3b00e-f887-484d-a8bb-f443bf09db1c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 09:00:29.719835 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-13a3b00e-f887-484d-a8bb-f443bf09db1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98f69056ca3e4e7c83f6c36c8e16c16c', 'request_id': 'req-13a3b00e-f887-484d-a8bb-f443bf09db1c'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 09:00:29.807836 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-13a3b00e-f887-484d-a8bb-f443bf09db1c admin admin] The token cache is being invalidate because project 98f69056ca3e4e7c83f6c36c8e16c16c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 09:00:29.808059 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-13a3b00e-f887-484d-a8bb-f443bf09db1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 09:00:29.808755 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-13a3b00e-f887-484d-a8bb-f443bf09db1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 09:00:29.810228 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1454/2903] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 09:00:29 2022] DELETE /identity/v3/projects/98f69056ca3e4e7c83f6c36c8e16c16c => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 09:00:29.814409 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542042b-1e42-4cfa-b929-d26e074e5a2b admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 09:00:29.815846 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-b542042b-1e42-4cfa-b929-d26e074e5a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 09:00:29.829826 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542042b-1e42-4cfa-b929-d26e074e5a2b admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 09:00:29.885705 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7879d9-a9a7-4570-9ca6-e4b4dbef41f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 09:00:29.887141 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7879d9-a9a7-4570-9ca6-e4b4dbef41f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 09:00:29.887374 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7879d9-a9a7-4570-9ca6-e4b4dbef41f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 09:00:29.887643 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7879d9-a9a7-4570-9ca6-e4b4dbef41f4 admin admin] PATH_INFO: `/v3/users/4449dce2c8b14df48b8746548e5ed55a` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 09:00:29.888873 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7879d9-a9a7-4570-9ca6-e4b4dbef41f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4449dce2c8b14df48b8746548e5ed55a)` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 09:00:29.890775 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7879d9-a9a7-4570-9ca6-e4b4dbef41f4 admin admin] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 09:00:29.896664 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 09:00:30.091443 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9e7879d9-a9a7-4570-9ca6-e4b4dbef41f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4449dce2c8b14df48b8746548e5ed55a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 09:00:30.092173 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9e7879d9-a9a7-4570-9ca6-e4b4dbef41f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4449dce2c8b14df48b8746548e5ed55a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 09:00:30.092772 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9e7879d9-a9a7-4570-9ca6-e4b4dbef41f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4449dce2c8b14df48b8746548e5ed55a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 09:00:30.104844 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9e7879d9-a9a7-4570-9ca6-e4b4dbef41f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4449dce2c8b14df48b8746548e5ed55a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 09:00:30.117154 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.notifications [None req-9e7879d9-a9a7-4570-9ca6-e4b4dbef41f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4449dce2c8b14df48b8746548e5ed55a'} {{(pid=80338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 09:00:30.220035 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1450/2904] 10.222.0.57 () {58 vars in 1223 bytes} [Mon Jul 18 09:00:29 2022] DELETE /identity/v3/users/4449dce2c8b14df48b8746548e5ed55a => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 09:00:30.466613 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3b00e-f887-484d-a8bb-f443bf09db1c admin admin] Authenticating user token {{(pid=80339) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 09:00:30.468189 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.fernet_utils [None req-13a3b00e-f887-484d-a8bb-f443bf09db1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80339) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 09:00:30.483748 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3b00e-f887-484d-a8bb-f443bf09db1c admin admin] Validating token access rules against request {{(pid=80339) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 09:00:30.545584 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d617c-0124-4ce9-8a9f-c0becd743555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ad24fe28ec40dbbca364cbdb34972d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eedba945f8d24629a712fbf156a2209f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80339) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 09:00:30.546459 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d617c-0124-4ce9-8a9f-c0becd743555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 09:00:30.546744 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d617c-0124-4ce9-8a9f-c0becd743555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 09:00:30.547027 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d617c-0124-4ce9-8a9f-c0becd743555 admin admin] PATH_INFO: `/v3/projects/392f7faedc094ec5aaf10f1961fa5300` {{(pid=80339) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 09:00:30.548293 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d617c-0124-4ce9-8a9f-c0becd743555 admin admin] RBAC: Authorizing `identity:delete_project(project_id=392f7faedc094ec5aaf10f1961fa5300)` {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 09:00:30.550263 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d617c-0124-4ce9-8a9f-c0becd743555 admin admin] RBAC: Authorization granted {{(pid=80339) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 09:00:30.574194 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 09:00:30.690882 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7f3d617c-0124-4ce9-8a9f-c0becd743555 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '392f7faedc094ec5aaf10f1961fa5300', 'request_id': 'req-7f3d617c-0124-4ce9-8a9f-c0becd743555'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 09:00:30.696617 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7f3d617c-0124-4ce9-8a9f-c0becd743555 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '392f7faedc094ec5aaf10f1961fa5300', 'request_id': 'req-7f3d617c-0124-4ce9-8a9f-c0becd743555'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 09:00:30.702779 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7f3d617c-0124-4ce9-8a9f-c0becd743555 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '392f7faedc094ec5aaf10f1961fa5300', 'request_id': 'req-7f3d617c-0124-4ce9-8a9f-c0becd743555'} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 09:00:30.790384 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7f3d617c-0124-4ce9-8a9f-c0becd743555 admin admin] The token cache is being invalidate because project 392f7faedc094ec5aaf10f1961fa5300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80339) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 09:00:30.790598 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7f3d617c-0124-4ce9-8a9f-c0becd743555 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 09:00:30.791215 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG keystone.notifications [None req-7f3d617c-0124-4ce9-8a9f-c0becd743555 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80339) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 09:00:30.792806 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: [pid: 80339|app: 0|req: 1455/2905] 10.222.0.57 () {58 vars in 1232 bytes} [Mon Jul 18 09:00:30 2022] DELETE /identity/v3/projects/392f7faedc094ec5aaf10f1961fa5300 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 09:00:40.216081 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 09:00:40.787166 dv-cinder-850142-3-iscsi devstack@keystone.service[80339]: DEBUG dbcounter [-] [80339] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80339) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 09:04:20.737398 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7879d9-a9a7-4570-9ca6-e4b4dbef41f4 admin admin] Authenticating user token {{(pid=80338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 09:04:20.738815 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-9e7879d9-a9a7-4570-9ca6-e4b4dbef41f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 09:04:20.885126 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7879d9-a9a7-4570-9ca6-e4b4dbef41f4 admin admin] Validating token access rules against request {{(pid=80338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 09:04:20.950995 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464a4e5-d4d7-4ec1-8b98-51e32ac9eacd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd606fd7ef269460a8110223932f02d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35c846931fd443fbb83a6fbe0aeaa866', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 09:04:20.952389 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464a4e5-d4d7-4ec1-8b98-51e32ac9eacd service placement] REQUEST_METHOD: `GET` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 09:04:20.952674 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464a4e5-d4d7-4ec1-8b98-51e32ac9eacd service placement] SCRIPT_NAME: `/identity` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 09:04:20.952947 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464a4e5-d4d7-4ec1-8b98-51e32ac9eacd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 09:04:20.954872 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.fernet_utils [None req-f464a4e5-d4d7-4ec1-8b98-51e32ac9eacd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 09:04:20.966197 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464a4e5-d4d7-4ec1-8b98-51e32ac9eacd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 09:04:20.967915 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464a4e5-d4d7-4ec1-8b98-51e32ac9eacd service placement] RBAC: Authorization granted {{(pid=80338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 09:04:20.980180 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: [pid: 80338|app: 0|req: 1451/2906] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Jul 18 09:04:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 09:04:30.973708 dv-cinder-850142-3-iscsi devstack@keystone.service[80338]: DEBUG dbcounter [-] [80338] Writing DB stats keystone:SELECT=89 {{(pid=80338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}